This change adds the ability to view various statistics for honeypot log data. This includes views such as unique SSH usernames, unique HTTP paths, unique HTTP host headers, etc.
A new `/logs/{logtype}/{subtype}` route is added for rendering stats.
This change adds support for WebSockets using Google's WebSocket package.
When the threat feed server is starting, a Go function is created to monitor honeypot log data via a channel. When log data is received on the channel, it is broadcast to all connected WebSocket clients.
A /live endpoint and handler is added for serving the live.html template. This page displays the log data in real time by utilizing WebSockets.
Updated the nav bar on all html pages to include the new 'Live' icon for accessing the realtime log.
This change adds a /config handler for displaying the Deceptifeed configuration. An icon is added to the nav bar for accessing the page.
- Add config.html template for displaying the Deceptifeed configuration.
- Add config.html supporting styles to style.css.
- Add /config http handler to render the config.html template.
- Add icon to nav bar for accessing /config page.
- Add a horizontal navigation bar to the top of all management pages.
- Tweaks to colors and styling of management pages.
- Rename the endpoint '/threatfeed' to '/webfeed'.
- Rename the file 'htmlfeed.html' to 'webfeed.html'.
- Remove the `/feed` endpoint, as this is already handled by `/plain`.
- Rename `/html` endpoint to `/threatfeed`.
- Add a `/docs` endpoint for displaying information on accessing the threat feed.
- Make the Deceptifeed logo smaller on HTML pages.
- Revise the layout of the initial homepage. Endpoint docs are now moved and linked to /docs.
- Remove `/html` from the list of threat feed endpoints in the docs, as it doesn't belong here (there is a link to the html threat feed on the home page).
This change updates the threat feed server so that the entire application config is passed in to the threat feed server. This allows the threat feed server to access its own configuration as well as the settings for each honeypot server and any other application-wide settings.
Refactor saveCSV to use a bufio.NewWriter and manually format CSV data rather than using the built-in CSV writer. This results in slightly better performance and memory usage when the threat feed is large (>500k entries).
Various miscellaneous changes:
- Rename `mutex` variable to `mu`.
- Lock the mutex in `loadCSV` because it accesses the iocData map.
This change removes the /stix/indicators and /stix/observables endpoints and replaces them with a single /stix endpoint. The /stix endpoint returns the threat feed as STIX indicators.
This change adds a default landing page that is served when accessing the threat feed root URL. The landing page provides information about accessing the threat feed.
Move the template from handleHTML to the templates directory and embed using embed.FS.
Several changes to HTTP routes:
- Delete `/csv/ips` route and handler.
- Delete `/json/ips` route and handler.
- Rename `/stix2` route to `/stix/indicators`.
- Rename `/stix2/ips` route to `/stix/observables`.
- Change default route `/` to return 404 Not Found.
- Move plain text feed to `/feed` and `/plain` routes.
- Add new route `/html` and associated handler for serving the threat feed as a web page for viewing in a browser.
- Revise sorting methods to apply a secondary sort by IP adddress when the values for the requested sort method are equal.
This change adds support for serving the threat feed over TAXII 2.1. The TAXII API is not yet fully implemented, but should work with most threat intelligence platforms.
Missing TAXII features:
- Mising `limit` query parameter.
- Missing `match` query parameter.
- Pagination is not supported.
- Missing `X-TAXII` headers.
- {api-root}/collections/{id}/manifest/
- {api-root}/collections/{id}/objects/{object-id}/
- {api-root}/collections/{id}/objects/{object-id}/versions/
- Add `internal\taxi\taxi.go` with struct definitions and helpers for TAXII resources.
- Move threat feed parsing functions from `handler.go` to `feed.go`.
- Add option functions for `prepareFeed` to allow sorting by last seen date and to filter by last seen date.
- Default timestamps to time.Now when initially loading feed data.
This change adds support for serving the threat feed in a STIX 2.1 format using the `/stix2` and `/stix2/ips` routes.
- Add `handleSTIX2` function for the `/stix2` route. This returns the detailed threat feed structured as a STIX bundle containing *Indicator* objects for each IP address in the feed.
- Add `handleSTIX2Simple` handler for the `/stix2/ips` route. This returns the simplified threat feed structured as a STIX bundle with each IP address included as a STIX Cyber-observable Object (SCO).
- Add `util.go` with helper functions for creating STIX 2.1 identifiers. This includes a `newUUIDv5` function, a `newUUIDv4` function, and a `namespace` type.
This change moves the threat feed's time.Ticker to a local variable. Previously, the variable was mistakenly declared as global. The time interval for the ticker is now declared as a const.
- Rename database.go to data.go.
- Move data-related global vars from threatfeed.go to data.go.
- Split out functions from threatfeed.go into seperate files:
- Move HTTP server functions to server.go.
- Move HTTP handler functions to handler.go.
- Move HTTP middleware functions to middleware.go.
- Rename hasMapChanged to dataChanged.
