Netgraph and Netflow-v9

[Kolasinski, Brent D.]

Hi All,

I have been doing some tests with the FreeBSD ng_netflow module for
netflow generation.  I am trying to export v9 netflow records to another
server running SiLK (which can receive v9 Netlfow from our Cisco routers
just fine).

When exporting v9 records from our FreeBSD-9-RELEASE server, we are
getting this error on our SiLK server (this repeats many times):
"rwflowpack[23113]: fBufNext: No Templates Present for Domain 0x000a"

Now I modified the settemplates variable in ngctl to send a template every
20 seconds, but we are still getting this.

As a sanity check, I tried exporting v5 netflow data from this FreeBSD box
to the Silk box, and it happily receives it and processes it.  The Silk
server is receiving the v9 netflow datagrams, as I can see it with a PCAP.

Any ideas as to what I am doing wrong?  Am I using the export9 hook
correctly in the commands listed below?  There is not much documentation
covering export9 out there (besides the tiny blurb in the FreeBSD9 Release
notes).

Here is a detail of my setup:
2 ethernet cards:
1) bce0 -> in promiscuous mode listening to traffic off of a tap
2) bce1 -> nic to be exporting netflow / connected to our network

Commands I am using to export v9 netflow records in ngctl:

mkpeer bce0: netflow lower iface0
name bce0:lower netflow
connect bce0: netflow: upper out0
mkpeer netflow: ksocket export9 inet/dgram/udp
msg netflow:export9 connect inet/<IP ADDRESS>:<PORT>


Thanks!!

----------
Brent Kolasinski
Cyber Security Program Office
Argonne National Laboratory
Phone: [phone]


_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"

[melifaro-2]

On 09.06.2012 00:04, Kolasinski, Brent D. wrote:

> Hi All,
>
> I have been doing some tests with the FreeBSD ng_netflow module for
> netflow generation.  I am trying to export v9 netflow records to another
> server running SiLK (which can receive v9 Netlfow from our Cisco routers
> just fine).
>
> When exporting v9 records from our FreeBSD-9-RELEASE server, we are
> getting this error on our SiLK server (this repeats many times):
> "rwflowpack[23113]: fBufNext: No Templates Present for Domain 0x000a"

>
> Now I modified the settemplates variable in ngctl to send a template every
> 20 seconds, but we are still getting this.
It should disappear after 5-10 minutes. We're using several FreeBSD v9
sensors with flowd and it seems to run fine (except first 5 minutes
while waiting for template). I'm aware about the problem with templates
timeout working incorrectly and I plan to fix this soon.

>
> As a sanity check, I tried exporting v5 netflow data from this FreeBSD box
> to the Silk box, and it happily receives it and processes it.  The Silk
> server is receiving the v9 netflow datagrams, as I can see it with a PCAP.
>
> Any ideas as to what I am doing wrong?  Am I using the export9 hook
> correctly in the commands listed below?  There is not much documentation
> covering export9 out there (besides the tiny blurb in the FreeBSD9 Release
> notes).
>
> Here is a detail of my setup:
> 2 ethernet cards:
> 1) bce0 ->  in promiscuous mode listening to traffic off of a tap
> 2) bce1 ->  nic to be exporting netflow / connected to our network
>
> Commands I am using to export v9 netflow records in ngctl:
>
> mkpeer bce0: netflow lower iface0
> name bce0:lower netflow
> connect bce0: netflow: upper out0
> mkpeer netflow: ksocket export9 inet/dgram/udp
> msg netflow:export9 connect inet/<IP ADDRESS>:<PORT>
>
>
> Thanks!!
>
> ----------
> Brent Kolasinski
> Cyber Security Program Office
> Argonne National Laboratory
> Phone: [phone]
>
>
> _______________________________________________
> [hidden email] mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "[hidden email]"
>

--
WBR, Alexander
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"

[Kolasinski, Brent D.]

On 6/9/12 5:01 AM, "Alexander V. Chernikov" <[hidden email]> wrote:

>It should disappear after 5-10 minutes. We're using several FreeBSD v9
>sensors with flowd and it seems to run fine (except first 5 minutes
>while waiting for template). I'm aware about the problem with templates
>timeout working incorrectly and I plan to fix this soon.

Looks like it has disappeared, however I am still not seeing any v9
collection.  I am assuming I am using export9 correctly in the ngctl
commands?

>
>>
>> Commands I am using to export v9 netflow records in ngctl:
>>
>> mkpeer bce0: netflow lower iface0
>> name bce0:lower netflow
>> connect bce0: netflow: upper out0
>> mkpeer netflow: ksocket export9 inet/dgram/udp
>> msg netflow:export9 connect inet/<IP ADDRESS>:<PORT>
>>
>>
>>
>
>--
>WBR, Alexander

Thanks

--Brent

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"

[melifaro-2]

On 11.06.2012 20:55, Kolasinski, Brent D. wrote:
>
> On 6/9/12 5:01 AM, "Alexander V. Chernikov"<[hidden email]>  wrote:
>
>> It should disappear after 5-10 minutes. We're using several FreeBSD v9
>> sensors with flowd and it seems to run fine (except first 5 minutes
>> while waiting for template). I'm aware about the problem with templates
>> timeout working incorrectly and I plan to fix this soon.
I've done some additional tests and it seems that templates are sent in
regular intervals exactly as specified in 'settemplate'.
However I still haven't tested this on real collector.
>
> Looks like it has disappeared, however I am still not seeing any v9
> collection.  I am assuming I am using export9 correctly in the ngctl
> commands?
It seems so.

Can you show "ngctl msg netflow: info" ouput ?

 > 1) bce0 -> in promiscuous mode listening to traffic off of a tap

Does bce0 have both UP and RUNNING flags set ?

>
>>
>>>
>>> Commands I am using to export v9 netflow records in ngctl:
>>>
>>> mkpeer bce0: netflow lower iface0
>>> name bce0:lower netflow
>>> connect bce0: netflow: upper out0
>>> mkpeer netflow: ksocket export9 inet/dgram/udp
>>> msg netflow:export9 connect inet/<IP ADDRESS>:<PORT>
>>>
>>>
>>>
>>
>> --
>> WBR, Alexander
>
>
> Thanks
>
> --Brent
>
>

--
WBR, Alexander
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"

[Kolasinski, Brent D.]

It appears that it may be something with my current collector.  While
debugging today, I decided to attempt to run Silk locally on the FreeBSD
netflow box.  

When exporting locally, it is reading the netflow-v9 records.  Yay!

Our collector is an older Linux box with a manually compiled current
version of Silk (not that it should matter which OS is running on the
collector) with the libfixbuf patch installed.  I wonder what is going on
there, alas, that is not your problem :)

Thanks for the help!

----------
Brent Kolasinski
Cyber Security Program Office
Argonne National Laboratory
Phone: [phone]




On 6/11/12 5:16 PM, "Kolasinski, Brent D." <[hidden email]> wrote:

>
>On 6/11/12 12:36 PM, "Alexander V. Chernikov" <[hidden email]>
>wrote:
>>
>>It seems so.
>>
>>Can you show "ngctl msg netflow: info" ouput ?
>
>Rec'd response "info" (805306369) from "[16]:":
>Args: { IPv4 bytes=4828162266587 IPv4 packets=1005674835 IPv4 records
>used=61793 fibs allocated=1 Active expiries=26901592 Inactive
>expiries=133410564 Inactive timeout=15 Active timeout=1800 }
>
>
>Now I am generating v5 netflow as well so I can compare - which I am
>seeing on the collector.  I can turn that off and just leave v9 on if that
>helps for debugging purposes.
>
>>
>> > 1) bce0 -> in promiscuous mode listening to traffic off of a tap
>>
>>Does bce0 have both UP and RUNNING flags set ?
>
>Yup.  Status is:
>
>bce0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC>
>metric 0 mtu 1500
> options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM
>,
>TSO4,VLAN_HWTSO,LINKSTATE>
> ether 00:19:b9:**:**:**
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
>
>
>--Brent
>

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[hidden email]"