Differences

This shows you the differences between two versions of the page.

--- app:auto_fingerprint [2017/11/29 11:30]
vivek created
+++ app:auto_fingerprint [2017/11/29 17:28] (current)
veera [Web Server Access Log]
@@ Line 2: / Line 2: @@
-TLS Fingerprinting is still in its early days therefore the coverage of known prints is not too deep.  The Trisul [[app:tlsfingerprint|TLS Fingerprint App]] ships with a known fingerprint database of about 500 entries but you will likely find anywhere from 50-70% unknown fingerprints. We need a way to resolve prints to client ID.
+TLS Fingerprinting is still in its early days therefore the coverage of known prints is not too deep.  The Trisul [[app:tlsfingerprint|TLS Fingerprint App]] ships with a known fingerprint database of about 500 entries ([[https://github.com/trisulnsm/trisul-scripts/blob/master/lua/frontend_scripts/reassembly/ja3/prints/ja3fingerprint.json|ja3prints.json]]) but you might find anywhere from 50-70% unknown fingerprints in your network. We need a way to resolve prints to client ID and build up the mappings.
-Internally we use a Ruby TRP script that can automate this process. The script is available at [[https://github.com/trisulnsm/trisul-scripts/tree/master/lua/frontend_scripts/reassembly/ja3/prints|mk_ja3print.rb]]
-===== How this works =====
+===== Resolving TLS Fingerprints =====
+Some of the techniques of resolving unknown fingerprints
+  - If you have access to a Web Server log  - Look at the ''User-Agent'' field
+  - If not, see if you can find a ''User-Agent'' vertex nearby, for example if the client followed a HTTP 302 Redirect to a https site.
+  - Look at Hosts using the fingerprint, see if you can detect a pattern or an application. Many web scanners, bots, and applications like Git, Dropbox can be detected this way.
+===== Web Server Access Log =====
+Internally we use a Ruby TRP script that can automate this process if given access to web server logs. The script is available on Github at [[https://github.com/trisulnsm/trisul-scripts/tree/master/lua/frontend_scripts/reassembly/ja3/prints|mk_ja3print.rb]]
+The script isnt too complicated. It works in the following way.
+  - Connects and gets list of unresolved JA3 TLS Prints in a 24 hour period.
+  - For Each unresolved print
+    - Use Graph Analytics to get list of Hosts using it
+    - Look for the Host in the Web Server Access log and pick out the first User-Agent
+    - Look for the print in the TLS Print App log - this contains the print and the Print String
+    - Print it out in JSON format
+Running the script.
+//Usage : mk_ja3fingerprint.rb  TRP-Server-Endpoint  Webserver-IP  Webserver-Access-Logs  Trisul-TLSPrint-Log-Pattern//
+A sample run of the script is shown below
+<code>
+$ ruby mk_ja3fingerprint.rb  tcp://74.207.234.90:12006  138.68.45.27    'trisul_access.log*'  'lua.stdout.jahash.lua.11*'
+"Found 29 Unresolved JA3 TLS Prints"
+"Sending EdgeGraph request vertex key=35c0a31c481927f022a3b530255ac080"
+"35c0a31c481927f022a3b530255ac080 resolved to  Mozilla/5.0 (compatible; RSiteAuditor)  771,49192-159-158-157-156-49195-49187-49191-49172-49171-61-60-53-47-49196-49188-49162-49161-106-64-56-50-10-19-5-4,65281-0-10-11-13-35,23-24,0"
+"Sending EdgeGraph request vertex key=37f691b063c10372135db21579643bf1"
+"37f691b063c10372135db21579643bf1 resolved to  urlgrabber/3.10 yum/3.4.3  771,49196-49162-49195-52393-49161-49200-49172-49199-52392-49171-159-57-56-107-158-52394-51-50-103-22-19-157-53-61-156-47-60-10-5-4,0-65281-10-11-13,29-23-24-25,0"
+"Sending EdgeGraph request vertex key=c2769dbd398f0b72e409887ceb9eb8ad"
+"Sending EdgeGraph request vertex key=05af1f5ca1b87cc9cc9b25185115607d"
+"05af1f5ca1b87cc9cc9b25185115607d resolved to  Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1  769,49172-49171-57-51-53-47-49162-49161-56-50-10-19-5-4,0-10-11-23-65281,23-24,0"
+"Sending EdgeGraph request vertex key=1885aa9927f99ed538ed895d9335995c"
+"1885aa9927f99ed538ed895d9335995c resolved to  Mozilla/55 (Windows NT 10.0; WOW64; rv:55.0) Gecko/20100101 Firefox/55  771,49195-49199-158-49162-49161-49171-49172-49159-49169-51-50-57-156-47-53-10-5-4-255,0-11-10-35-13-15,14-13-25-11-12-24-9-10-22-23-8-6-7-20-21-4-5-18-19-1-2-3-15-16-17,0-1-2"
+"Sending EdgeGraph request vertex key=05e15a226e00230c416a8cdefeb483c7"
+"Sending EdgeGraph request vertex key=1543a7c46633acf71e8401baccbd0568"
+"Sending EdgeGraph request vertex key=f22bdd57e3a52de86cda40da2d84e83b"
+..
+"Output written to file /tmp/fingerprint.json"
+</code>
+Once the script is finished, the JSON output is written to ''/tmp/prints.json'' this can be easily appended to the TLS Prints database.
+<code json>
+{"desc":"RSiteAuditor","ja3_hash":"35c0a31c481927f022a3b530255ac080","ja3_str":"771,49192-159-158-157-156-49195-49187-49191-49172-49171-61-60-53-47-49196-49188-49162-49161-106-64-56-50-10-19-5-4,65281-0-10-11-13-35,23-24,0"}
+{"desc":"urlgrabber/3.10 yum/3.4.3","ja3_hash":"37f691b063c10372135db21579643bf1","ja3_str":"771,49196-49162-49195-52393-49161-49200-49172-49199-52392-49171-159-57-56-107-158-52394-51-50-103-22-19-157-53-61-156-47-60-10-5-4,0-65281-10-11-13,29-23-24-25,0"}
+{"desc":"Feedly/1.0","ja3_hash":"f22bdd57e3a52de86cda40da2d84e83b","ja3_str":"771,49188-49192-61-49190-49194-107-106-49162-49172-53-49157-49167-57-56-49187-49191-60-49189-49193-103-64-49161-49171-47-49156-49166-51-50-49196-49195-49200-157-49198-49202-159-163-49199-156-49197-49201-158-162-49160-49170-10-49155-49165-22-19-255,10-11-13-0,23-24-25-9-10-11-12-13-14-22,0"}
+</code>
+Iteratively running this script for a few days can resolve most of the unknown prints. That makes outlier detection much easier.
+===== Other methods to resolve =====
+Once you get the unknown prints down to 10-20% you can use Trisul's excellent Graph Analytics manually to explore and nail down each print.  We will see that in another article.

Trisul Network Analytics Developer Zone

User Tools

Site Tools

Differences

Page Tools

Trisul Network Analytics
Developer Zone