Jay Taylor's notes

back to listing index

Human readable format for http headers with tcpdump

[web search]
Original source (serverfault.com)
Tags: tcpdump traffic-sniffing one-liners serverfault.com
Clipped on: 2015-10-07

I would like to view the HTTP headers sent from Apache (listening on port 80) to Tomcat (on port 4080) in a Linux machine.

According to Wikipedia,

Header fields are colon-separated name-value pairs in clear-text string format.

I've tried some variations of the following tcpdump command:

$ sudo tcpdump -lnX dst port 4080 -c 10

11:29:28.605894 IP SOME_IP.33273 > SOME_IP.4080: P 0:49(49) ack 1 win 23 <nop,nop,timestamp 1191760962 509391143>
    0x0000:  4500 0065 3a9f 4000 3f06 0084 628a 9ec4  E..e:.@.?...b...
    0x0010:  628a 9c97 81f9 0ff0 9e87 eee0 144b 90e1  b............K..
    0x0020:  8018 0017 fb43 0000 0101 080a 4708 d442  .....C......G..B
    0x0030:  1e5c b127 4845 4144 202f 6461 7070 6572  .\.'HEAD./dapper
    0x0040:  5f73 6572 7669 6e67 2f41 644d 6f6e 6b65  _serving/AdMonke
    0x0050:  793f                                     y?

The result was always the same - a strange mix of gibberish and English words (e.g. HEAD).

How can I view the headers in a human-readable format?

asked May 2 '13 at 11:41
Image (Asset 2/11) alt=
Adam Matan
1,86072344
   upvote
  flag
Tcpdump shows the entire packet. This includes the IP, and TCP headers. AFAIK, you can't display just the TCP payload. –  Zoredache May 2 '13 at 15:29

Here's a one-liner I came up with for displaying request and response HTTP headers using tcpdump (which should work for your case too):

sudo tcpdump -A -s 10240 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' | egrep --line-buffered "^........(GET |HTTP\/|POST |HEAD )|^[A-Za-z0-9-]+: " | sed -r 's/^........(GET |HTTP\/|POST |HEAD )/\n\1/g'

It limits cuts the packet off at 10Kb and only knows GET, POST and HEAD commands, but that should be enough in the majority of cases.

EDIT: modified it to get rid of the buffers at every step to make it more responsive. Needs Perl and stdbuf now though, so use the original version if you don't have those:

sudo stdbuf -oL -eL /usr/sbin/tcpdump -A -s 10240 "tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)" | egrep -a --line-buffered ".+(GET |HTTP\/|POST )|^[A-Za-z0-9-]+: " | perl -nle 'BEGIN{$|=1} { s/.*?(GET |HTTP\/[0-9.]* |POST )/\n$1/g; print }'

Some explanations:

  • sudo stdbuf -oL -eL makes tcpdump run line-buffered
  • the tcpdump magic filter is explained in detail here: http://stackoverflow.com/questions/11757477/understanding-tcpdump-filter-bit-masking
  • grep is looking for any lines with GET, HTTP/ or POST; or any lines that look like a header (letters and numbers followed by colon)
  • BEGIN{$|=1} causes perl to run line-buffered
  • s/.*?(GET |HTTP/[0-9.]* |POST )/\n$1/g adds a newline before the beginning of every new request or response
answered Oct 3 '14 at 22:37
Image (Asset 3/11) alt=
Kibber
18113
   upvote
  flag
Works great. Could you please add more details on how that tcpdump expression work? –  Vivek Thomas Oct 4 '14 at 4:54
1 upvote
  flag
the 'ip' part in parens is explained here, for example: stackoverflow.com/questions/11757477/… –  Kibber May 27 at 1:11
   upvote
  flag
You just saved me so much headache. Shame i can only +1. –  Aaron Dobbing Sep 10 at 13:01

You can get something close to what you want by using -A, e.g.

E....c@.@...
.....Ng.d.P..Ch.).....s.......
.A...u.BHEAD / HTTP/1.1
User-Agent: curl/7.29.0
Host: www.google.com
Accept: */*

Remember to use -s 0 to make sure you get the whole packet.

Alternatively you could use wireshark to view the headers interactively.

answered May 2 '13 at 11:48
Image (Asset 4/11) alt=
Flup
3,65511232
1 upvote
  flag
Tried -A and -s 0, got the same output. –  Adam Matan May 2 '13 at 11:51
1 upvote
  flag
Try without -X. –  Flup May 2 '13 at 11:52
   upvote
  flag
tcpdump -s 0 -A dst port 4080 gives E..e..@.?.$bb...b....:......w........Q.....G..1.b..HEAD /dapper_serving/AdMonkey?ping=1 HTTP/1.0. –  Adam Matan May 2 '13 at 11:56
   upvote
  flag
...which is something close to what you want. Read from 'HEAD' -- this is the HTTP payload. If you've definitely used -s 0 and there's nothing after HTTP/1.0, there are no HTTP headers in the request. –  Flup May 3 '13 at 7:00
   upvote
  flag
thanks. Is there a way to print only the text headers, sans the binary payload? –  Adam Matan May 3 '13 at 12:47

Try to use http://justniffer.sourceforge.net/ It is better tool or Wireshark with "Follow TCP Flow" option, there are just many better options than tcpdump to see headers (requests/responses)

answered May 2 '13 at 11:51
Image (Asset 5/11) alt=
Danila Ladner
3,429622

Your Answer

asked

2 years ago

viewed

18802 times

active

25 days ago

Hot Network Questions

Technology Life / Arts Culture / Recreation Science Other
  1. Stack Overflow
  2. Server Fault
  3. Super User
  4. Web Applications
  5. Ask Ubuntu
  6. Webmasters
  7. Game Development
  8. TeX - LaTeX
  1. Programmers
  2. Unix & Linux
  3. Ask Different (Apple)
  4. WordPress Development
  5. Geographic Information Systems
  6. Electrical Engineering
  7. Android Enthusiasts
  8. Information Security
  1. Database Administrators
  2. Drupal Answers
  3. SharePoint
  4. User Experience
  5. Mathematica
  6. Salesforce
  7. ExpressionEngine® Answers
  8. more (13)
  1. Photography
  2. Science Fiction & Fantasy
  3. Graphic Design
  4. Movies & TV
  5. Seasoned Advice (cooking)
  6. Home Improvement
  7. Personal Finance & Money
  8. Academia
  9. more (9)
  1. English Language & Usage
  2. Skeptics
  3. Mi Yodeya (Judaism)
  4. Travel
  5. Christianity
  6. Arqade (gaming)
  7. Bicycles
  8. Role-playing Games
  9. more (21)
  1. Mathematics
  2. Cross Validated (stats)
  3. Theoretical Computer Science
  4. Physics
  5. MathOverflow
  6. Chemistry
  7. Biology
  8. more (5)
  1. Stack Apps
  2. Meta Stack Exchange
  3. Area 51
  4. Stack Overflow Careers
site design / logo © 2015 Stack Exchange Inc; user contributions licensed under cc by-sa 3.0 with attribution required
rev 2015.10.7.2870