24 Commits

Author SHA1 Message Date
Vincent Bernat
a61288eb19 decoders: fix sflow parsing of IP and MAC addresses (#261)
Due to IP and MAC addresses being a non-standard type, utils.BinaryRead
was not able to decode them. Move these two types inside utils.go and
teach BinaryRead to use them.

Co-authored-by: lspgn <lspgn@users.noreply.github.com>
2024-01-06 00:59:53 -08:00
Louis
4f053d338f decoders: do not stop reading sets (#239) 2023-12-09 11:56:17 -08:00
Louis
a1750f5f09 decoder: ipfix have max size (#237) 2023-12-01 00:40:10 -08:00
Louis
76246841fb decoders: add test/benchmark (#232) 2023-11-04 16:04:47 -07:00
Louis
00c77d7605 decoders: improve BasicTemplateSystem (#233)
* Use write lock when removing template

Since template removal modifies the template set map, it should be done
while holding a write lock, instead of a read lock.

* Optimize BasicTemplateSystem

The BasicTemplateSystem that comes with the library uses a 3-level map
to hold template information to avoid templateId and obsDomainId
collision between multiple protocols and hosts. However, the same can be
done by using a single map with a 64-bit key consisting of version,
templateId and obsDomainId.

This greatly simplifies the code and reduces the number of map lookups
from 3 to 1 per call to GetTemplate.

Co-authored-by: Matheus Castanho <matheus.castanho@dcc.ufmg.br>
2023-11-04 16:04:36 -07:00
Thibaud Cheruy
2a8f999a95 decoder: fix netflow flowsets decoding (#218)
* decoder was only reading the first FlowSet of those messages, resulting in unaccounted flows

Co-authored-by: lspgn <lspgn@users.noreply.github.com>
2023-09-28 20:11:18 -07:00
Louis
34a0c1618e decoders: port reader changes from v1 to v2 (#204) 2023-09-04 03:19:41 -07:00
Louis
a6fe3ffb12 producer: bugfix of sampling mapping for ipfix (#207) 2023-08-23 20:51:54 -07:00
Louis
ae56e41786 refactor: v2 (#150) 2023-08-09 19:47:20 -07:00
Louis
e5696f114b decoders: fix netflow legacy/v5 sampling interval (#171) 2023-05-25 06:10:28 -07:00
Louis
69a6eaf99e templates: retrofit to avoid issue when upgrading dep (#160) 2023-04-20 20:05:33 -07:00
Louis
958109e91c allow unknown records for sFlow (#155) 2023-04-15 21:21:10 -07:00
Louis
9850000ffa allow unknown record structures in sFlow (#153) 2023-04-15 20:59:53 -07:00
Louis
c4c4ffb4e4 Refactor template as module (#49)
* Refactor template as module
* Abstraction to allow custom template storage (eg: file, redis, http...)
* Works similarly to Transport and Format
2023-04-15 20:52:32 -07:00
lspgn
99e086f684 support netflow v9 variable length 2022-04-02 10:40:35 -07:00
Vincent Bernat
e3f8f990ec Add some tests for Netflow decoding
The tests are a bit more expansive than the existing tests for sFlow
or NFv5 as we check the whole structure. I am also testing the
String() function as it is easier to read. It is a bit redundant, but
checking only for the wire format makes it difficult to compare with
Wireshark. Only testing for the textual representation is not totally
good as it is not what is used by users of the decode function.
2022-02-26 16:47:08 +01:00
Mario Macias
615b9f697c Avoid unnecessary memory allocations 2022-01-26 17:24:35 +01:00
lspgn
43cf8b58d9 Bugfix: decoding OptionsTemplateSet
* Was improperly decoding scope fields
* Start removing custom errors structures (will optimize template not found)
2021-11-13 15:49:27 -08:00
Vincent Bernat
17a96d9911 netflow: correctly decode options template set (#39)
netflow: correctly decode options template set
2021-09-23 20:46:39 -07:00
Louis
b0b73b2b90 Merge pull request #38 from vincentbernat/fix/defer-unlock
style: defer unlock when possible/not trivial
2021-09-23 20:44:05 -07:00
Louis
536b08812f Custom map flow fields (#36)
* adds dataframe link decoding
* can map NetFlow/IPFIX fields and bytes sections from sFlow/packets to any field inside the protobuf
* add CLI argument for loading a mapping yaml file
2021-09-23 20:41:17 -07:00
Vincent Bernat
95945d3042 style: defer unlock when possible/not trivial
Defer unlocking just after taking a lock when possible (when unlock is
done at the very end) and when not trivial (the function body is more
than a couple of lines). This simplifies a bit some functions (no need
to unlock before each return) and for the other, it may avoid a bug in
the future in case a return is inserted into the body of a function.

Use of defer has been optimized a lot in Go and it is believed that
simpler defers have zero overhead since Go 1.14:
https://golang.org/doc/go1.14#runtime

> This release improves the performance of most uses of defer to incur
> almost zero overhead compared to calling the deferred function
> directly. As a result, defer can now be used in performance-critical
> code without overhead concerns.
2021-09-23 10:01:24 +02:00
lspgn
1fda65fe41 bugfix: variable length, missing option templates 2021-05-25 21:52:59 -07:00
lspgn
e4bacbc4bd init 2021-05-25 17:33:26 -07:00