[PATCH] Script to summarise tshark output

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH] Script to summarise tshark output

Gary Lockyer
Script to provide an anonymous summary from tshark

The tshark command needs to output a PDML XML stream, which this command
will read. The summary is intended not to expose private or customer
data while allowing a good view on the range and frequency of the
network traffic.

0001-script-Add-test-data-for-traffic_summary.pl.patch (672K) Download Attachment
0002-script-Add-script-to-provide-an-anonymous-summary-fr.patch (26K) Download Attachment
signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Script to summarise tshark output

Gary Lockyer
This email did not appear on the list, perhaps due to size.

Patches are available in the following repo
http://git.catalyst.net.nz/gw?p=samba.git;a=shortlog;h=refs/heads/traffic-summary


On 17/02/17 11:11, Gary Lockyer wrote:
> Script to provide an anonymous summary from tshark
>
> The tshark command needs to output a PDML XML stream, which this command
> will read. The summary is intended not to expose private or customer
> data while allowing a good view on the range and frequency of the
> network traffic.
>


signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] Script to summarise tshark output

Andrew Bartlett
On Fri, 2017-02-17 at 15:23 +1300, Gary Lockyer wrote:

> This email did not appear on the list, perhaps due to size.
>
> Patches are available in the following repo
> http://git.catalyst.net.nz/gw?p=samba.git;a=shortlog;h=refs/heads/tra
> ffic-summary
>
>
> On 17/02/17 11:11, Gary Lockyer wrote:
> > Script to provide an anonymous summary from tshark
> >
> > The tshark command needs to output a PDML XML stream, which this
> > command
> > will read. The summary is intended not to expose private or
> > customer
> > data while allowing a good view on the range and frequency of the
> > network traffic.
> >
G'Day Gary,

This looks really, really helpful!  

Reviewed-by: Andrew Bartlett <[hidden email]>

Can I get a second team review for this?

Also, if I can get some folks to run this on their networks, we would
really like to know better what the traffic load on real-world Samba
and Windows domains look like.

Thanks,

Andrew Bartlett

signature.asc (879 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Call for samples: Please help us build a Samba AD performance measuring tool

Andrew Bartlett
In reply to this post by Gary Lockyer
On Fri, 2017-02-17 at 11:11 +1300, Gary Lockyer wrote:
> Script to provide an anonymous summary from tshark
>
> The tshark command needs to output a PDML XML stream, which this
> command
> will read. The summary is intended not to expose private or customer
> data while allowing a good view on the range and frequency of the
> network traffic.

The script Gary posted, which is available from 

http://git.catalyst.net.nz/gitweb?p=samba.git;a=blob_plain;f=script/traffic_summary.pl;hb=4786141c9d29d9eaaab3809542ee97af33db6285

is one step towards building a Samba performance montoring tool.  

We hope to start on that tool soon, which will take these summaries of
real-world network traffic and generate a synthetic load. 

Cramming the network full of packets is easy, but it would be really
helpful if folks running Samba or Windows AD networks could run this
script over a pcap file captured on your DCs so we get a good idea what
'real world' looks like.

The instructions are in the perl script, they involve using tshark to
parse the .pcap into XML, which this tool ingests, returning anonymous
lines of traffic summaries that we can build new traffic simulations
from.

The resulting files are nothing like the size of the .pcap but can be a
little large, but are totally anonymous and trivial to audit so we can
probably keep them in the wiki or somewhere for folks who are
comfortable.  

If you want to help, then please e-mail the output to me with some
details on your network size to me in the meantime if you can.

Thanks,

Andrew Bartlett

signature.asc (879 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Call for samples: Please help us build a Samba AD performance measuring tool

Andrew Bartlett
On Fri, 2017-02-24 at 18:22 +1300, Andrew Bartlett wrote:

> On Fri, 2017-02-17 at 11:11 +1300, Gary Lockyer wrote:
> > Script to provide an anonymous summary from tshark
> >
> > The tshark command needs to output a PDML XML stream, which this
> > command
> > will read. The summary is intended not to expose private or
> > customer
> > data while allowing a good view on the range and frequency of the
> > network traffic.
>
> The script Gary posted, which is available from 
>
> http://git.catalyst.net.nz/gitweb?p=samba.git;a=blob_plain;f=script/t
> raffic_summary.pl;hb=4786141c9d29d9eaaab3809542ee97af33db6285

For those reading the archives, it is now in master: 

https://git.samba.org/?p=samba.git;a=blob_plain;f=script/traffic_summary.pl;hb=HEAD

> is one step towards building a Samba performance montoring tool.  
>
> We hope to start on that tool soon, which will take these summaries
> of
> real-world network traffic and generate a synthetic load. 
>
> Cramming the network full of packets is easy, but it would be really
> helpful if folks running Samba or Windows AD networks could run this
> script over a pcap file captured on your DCs so we get a good idea
> what
> 'real world' looks like.
>
> The instructions are in the perl script, they involve using tshark to
> parse the .pcap into XML, which this tool ingests, returning
> anonymous
> lines of traffic summaries that we can build new traffic simulations
> from.
>
> The resulting files are nothing like the size of the .pcap but can be
> a
> little large, but are totally anonymous and trivial to audit so we
> can
> probably keep them in the wiki or somewhere for folks who are
> comfortable.  
>
> If you want to help, then please e-mail the output to me with some
> details on your network size to me in the meantime if you can.
>
> Thanks,
>
> Andrew Bartlett
--
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba