paulmataruso/goflow2
1
0
Fork 0
mirror of https://github.com/openobserve/goflow2.git synced 2025-11-02 13:03:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
58 Commits 1 Branch 2 Tags
615b9f697cc23c22a2181adfde1005c471f62347
Commit Graph

58 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mario Macias
615b9f697c Avoid unnecessary memory allocations 2022-01-26 17:24:35 +01:00
Louis
8d59905c44 Merge pull request #68 from netsampler/bugfix/enricher-2 
Bugfix: issues when reading a partial chunk of protobuf from stdin
 2022-01-24 20:34:11 -08:00
lspgn
7acb84835b Bugfix: issues when reading a partial chunk of protobuf from stdin 2022-01-24 20:30:30 -08:00
Louis
20e8e56709 Merge pull request #65 from netsampler/bugfix/offset 
Bugfix: index out of range in some cases
 2022-01-06 22:00:10 +01:00
lspgn
27863691f0 Bugfix: index out of range in some cases 2022-01-06 21:59:02 +01:00
Louis
4308c483a4 Merge pull request #61 from netsampler/feature/upgrade-sarama 
upgrade sarama to 1.30.1
 2021-12-27 14:20:55 +01:00
lspgn
05b436277b upgrade sarama to 1.30.1 2021-12-27 14:17:43 +01:00
Louis
a8a50f785c Merge pull request #60 from netsampler/feature/add-sctp 
Map ProtoName for SCTP
 2021-12-20 17:06:40 +01:00
lspgn
def05e071b Map ProtoName for SCTP 2021-12-20 16:53:00 +01:00
Louis
2e1cf5bbfd Merge pull request #48 from netsampler/feature/kafka-flush-control 
Add flags to control Kafka Flush parameters
 2021-11-13 16:41:38 -08:00
Louis
de5e751729 Merge pull request #53 from netsampler/bugfix/optiontemplates 
Bugfix: decoding OptionsTemplateSet
 2021-11-13 16:41:32 -08:00
lspgn
43cf8b58d9 Bugfix: decoding OptionsTemplateSet 
* Was improperly decoding scope fields
* Start removing custom errors structures (will optimize template not found)
 2021-11-13 15:49:27 -08:00
Louis
bf66556023 Merge pull request #52 from vincentbernat/fix/non-terminated-stream 
utils: copy payload before accepting another UDP packet
 2021-11-10 08:10:39 -08:00
Vincent Bernat
0f5528170d utils: copy payload before accepting another UDP packet 
In d1e1ace318 ("Allow Flow Routines to be cancellable (#40)"), the
payload was passed to another goroutine and erased by the next packet
to be received if the goroutine did not process it fast enough. Make
a copy before passing it to the goroutine to fix that.
 2021-11-10 15:56:49 +01:00
Louis
8fd64006ef Merge pull request #51 from leoluk/remove-kafkastate 
Remove unused KafkaState
 2021-11-05 09:16:34 -07:00
Leopold Schabel
56a2ffa46b Remove unused KafkaState 2021-11-03 16:59:57 +01:00
Mario Macias
d1e1ace318 Allow Flow Routines to be cancellable (#40) 
* Allow Flow Routines to be cancellable
 2021-10-31 16:42:07 -07:00
lspgn
c145be66cd Add flags to control Kafka Flush parameters 2021-10-31 10:23:03 -07:00
Louis
92043a6233 Merge pull request #47 from netsampler/cleanup/mod-extension 
cleanup: bump to go 1.17, sarama to 1.30.0, fix extensions
 2021-10-30 19:51:12 -07:00
lspgn
ec08b786c8 cleanup: bump to go 1.17, sarama to 1.30.0, fix extensions 2021-10-30 19:47:11 -07:00
Louis
7baa828267 Merge pull request #45 from shyam334/update-dep-sarama 
deps: bump sarama to v1.29.1
 2021-10-30 19:40:58 -07:00
Shyam Sundar
a9f9289f6b deps: bump sarama to v1.29.1 2021-10-28 15:06:42 +11:00
Vincent Bernat
17a96d9911 netflow: correctly decode options template set (#39) 
netflow: correctly decode options template set
 2021-09-23 20:46:39 -07:00
Louis
b0b73b2b90 Merge pull request #38 from vincentbernat/fix/defer-unlock 
style: defer unlock when possible/not trivial
 2021-09-23 20:44:05 -07:00
Louis
536b08812f Custom map flow fields (#36) 
* adds dataframe link decoding
* can map NetFlow/IPFIX fields and bytes sections from sFlow/packets to any field inside the protobuf
* add CLI argument for loading a mapping yaml file
 2021-09-23 20:41:17 -07:00
Vincent Bernat
95945d3042 style: defer unlock when possible/not trivial 
Defer unlocking just after taking a lock when possible (when unlock is
done at the very end) and when not trivial (the function body is more
than a couple of lines). This simplifies a bit some functions (no need
to unlock before each return) and for the other, it may avoid a bug in
the future in case a return is inserted into the body of a function.

Use of defer has been optimized a lot in Go and it is believed that
simpler defers have zero overhead since Go 1.14:
https://golang.org/doc/go1.14#runtime

> This release improves the performance of most uses of defer to incur
> almost zero overhead compared to calling the deferred function
> directly. As a result, defer can now be used in performance-critical
> code without overhead concerns.
 2021-09-23 10:01:24 +02:00
Louis
defd786b2a Merge pull request #34 from jotak/podman 
Makefile: parameterized DOCKER_REPO and DOCKER_BIN
 2021-09-09 21:51:39 -07:00
Louis
13daf3b669 Merge pull request #33 from amorenoz/delta 
ipfix: support relative flow timestamps
 2021-09-09 21:51:12 -07:00
Joel Takvorian
d3e3fdb0f8 Makefile: parameterized DOCKER_REPO and DOCKER_BIN 
e.g. allowing to build with podman

Also add gitignore for dist/

Signed-off-by: Joel Takvorian <jtakvori@redhat.com>
 2021-09-09 14:53:26 +02:00
Adrian Moreno
9247511c44 ipfix: support relative flow timestamps 
IPFIX supports sending flowEndDeltaMicroseconds (159) and
flowEndDeltaMicroseconds (160) to provide flow timestamps relative to
the exportTime in the IPFIX Message Header.

Use them to calculate flow TimeFlowStart and TimeFlowEnd.

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
 2021-09-08 17:12:31 +02:00
Louis
11acc67852 Merge pull request #32 from loganmc10/patch-1 
Fix README for SrcPort and DstPort
 2021-09-07 08:43:42 -07:00
Logan
5ad65d5100 Fix README for SrcPort and DstPort 2021-09-07 07:35:09 -06:00
Louis
faca667b19 Merge pull request #30 from netsampler/bug/enricher-decode 
bugfix: enricher decoding certain protobuf
 2021-08-29 08:00:47 -07:00
Louis
0b0f9dd952 Merge pull request #25 from netsampler/feature/enricher-sampling 
Enricher can set sampling rate
 2021-08-28 10:38:07 -07:00
lspgn
21c7fee62e bugfix: enricher decoding certain protobuf 
* now requires length prefixed messages
* allows custom line separator for text transport output
 2021-08-27 10:04:22 -07:00
Louis
a645df2da4 Merge pull request #27 from netsampler/feature/golang-117 
bump golang version to 1.17
 2021-08-16 18:43:49 -07:00
lspgn
84dc1ff5d7 bump golang version to 1.17 2021-08-16 18:37:42 -07:00
lspgn
5a42b2b1f7 Enricher can set sampling rate 2021-08-10 22:32:43 -07:00
Louis
1ad90a7375 Create LICENSE 2021-08-10 09:34:58 -07:00
Louis
8677c8d9ba Merge pull request #23 from s-chekanov/fix_db_start 
Fix clickhouse startup after restart
 2021-08-02 07:38:41 -07:00
Sergei Chekanov
3384a90a29 Fixing clickhouse startup after restart 2021-08-02 15:33:41 +03:00
Louis
6465eeb200 Merge pull request #17 from netsampler/feature/arm 
feature: add arm builds
 2021-06-11 23:38:57 -07:00
lspgn
583059057a feature: add arm builds 2021-06-11 23:30:56 -07:00
Louis
e52a053c9c formatting improved with selectors and text output (#14) 
* Improve format registration
* less interleaving (json does not require importing protobuf)
* generic text renderer
* escape for strings in text/json formatter
 2021-06-11 08:42:32 -07:00
Louis
9bdcdf0b22 Merge pull request #16 from netsampler/bugfix/destination-mac-ipfix 
Bugfix NetFlow/IPFIX producer
 2021-06-10 18:59:33 -07:00
lspgn
04c8543306 Bugfix NetFlow/IPFIX producer 
* Source and destination mac addresses are fed from `NFV9_FIELD_xx_yyy_MAC`
  Fixes a bug where the samples were missing either source or destination Mac.
  This is supposed to be linked to the sampling direction.
 2021-06-09 18:08:55 -07:00
Louis
88cbf58172 Merge pull request #10 from simPod/mf-lf 
Ensure line feed in Makefile
 2021-06-07 23:21:29 -07:00
Simon Podlipsky
7da5c4da27 Ensure line feed in Makefile 2021-06-08 08:07:52 +02:00
Louis
645497bec6 Merge pull request #9 from netsampler/fix/protobuf 
Fix protobuf
 2021-06-07 22:50:42 -07:00
lspgn
ab81e52b43 Fix protobuf 
* Rebuiild with protoc v3.17.2 and protoc-gen-go v1.26.0
* Add misisng go_package flag
* Add protobuf documentation
 2021-06-07 22:43:09 -07:00