This change removes the `CustomThreatsPath` setting from the threat feed configuration. The default configuration files are updated with this setting removed.
This change allows for comments in the exclude list using the `#` symbol. The `#` symbol on a line and everything after it is ignored when parsing the exclude list.
- Add `Version` string var to config package for storing version information.
- Update Makefile to set the `Version` variable at build time using the latest Git tag.
- Add `-version` flag to main package to print version information and exit.
- Remove setting the GO111MODULE environment variable from Makefile when building. It's not needed.
This change removes the 'threat score' feature which allowed users to configure each honeypot server with a variable 'score' when updating the threat feed.
It is replaced with a fixed observation count that is incremented by 1 for each honeypot interaction.
The field `threat_score` has been replaced with `observations` in all API call parameters and threat feed data.
The `threat_score` field in the CSV file has been renamed to `observations`. Existing threat feed CSV files will be automatically updated on the next threat feed save.
This change explicitly sets the default SSH banner in the running configuration when the application starts.
If starting without a configuration file, the global default banner is always used.
If starting with a configuration file, the global default banner is used when no banner is specified.
Adjust the html templates for web feed and log viewers to check if there's any data to display. If no data, a message is shown informing the user that there's no data.
Previously:
When no log path was specified for a server, it would fall back to the global/default log path during logger initialization. However, the `LogPath` field didn't update and would not reflect the actual path used by the logger.
Now:
Log path determination is handled while the configuration is loading. If a server falls back to the default log path, the `LogPath` field is updated to reflect the actual path used by the logger.
- Add a horizontal navigation bar to the top of all management pages.
- Tweaks to colors and styling of management pages.
- Rename the endpoint '/threatfeed' to '/webfeed'.
- Rename the file 'htmlfeed.html' to 'webfeed.html'.
- Remove the `/feed` endpoint, as this is already handled by `/plain`.
- Rename `/html` endpoint to `/threatfeed`.
- Add a `/docs` endpoint for displaying information on accessing the threat feed.
- Make the Deceptifeed logo smaller on HTML pages.
- Revise the layout of the initial homepage. Endpoint docs are now moved and linked to /docs.
- Remove `/html` from the list of threat feed endpoints in the docs, as it doesn't belong here (there is a link to the html threat feed on the home page).
As of Go 1.24, rand.Read is guaranteed to never return an error. This change removes the error check and the associated fallback function when calling rand.Read.
This change updates the threat feed server so that the entire application config is passed in to the threat feed server. This allows the threat feed server to access its own configuration as well as the settings for each honeypot server and any other application-wide settings.
Refactor saveCSV to use a bufio.NewWriter and manually format CSV data rather than using the built-in CSV writer. This results in slightly better performance and memory usage when the threat feed is large (>500k entries).
Various miscellaneous changes:
- Rename `mutex` variable to `mu`.
- Lock the mutex in `loadCSV` because it accesses the iocData map.
This change configures the honeypot loggers to use the internal `logrotate` package. This enables automatic log file rotation. The maximum log size is set to 50 MB.
The logrotate package implements an io.WriteCloser with automatic file rotation. When the file size exceeds a specified limit, the file is renamed with a `.1` suffix. Subsequent rotations overwrite the file with the `.1` suffix, meaning only one backup is retained. This approach keeps the rotation simple.
This change adjusts logger initialization to ensure only 1 file handle and 1 logger is created per unique log path.
Each honeypot server may have its own log path specified, which may be unique or may be shared with other honeypot servers.
Previously, each honeypot server would open a separate file handle and logger, even if the file was already opened by another server.
This change adjusts the default SSH server version string to `SSH-2.0-OpenSSH_9.6`. This makes SSH honeypots appear as OpenSSH 9.6.
The change is applied both to default configuration files and as an application default when no configuration is provided.
This change adjusts the SSH, TCP, and UDP honeypots to print quoted (escaped) strings to the terminal for certain log fields rather than raw strings. The adjusted fields are SSH username, SSH password, TCP responses, and UDP received data.
Changes to HTML feed template:
- Drop milliseconds from added and last seen dates.
- Make the logo a link to return to the threat feed homepage.
- Fix logo sizing on small screens.
- Reduce logo margins.
Changes to home page template:
- Update /stix API endpoint documentation to change references from /stix/indicators and /stix/observables to just /stix.
- Update TAXII documentation to include the new sightings collection.
- Update TAXII documentation with shortened collection names.
- Set `break-word` wrapping on code snippets to fix rendering on small screens.
- Collapse API endpoint table on small screens.
- Fix logo sizing on small screens.
- Reduce logo margins.
This change removes the /stix/indicators and /stix/observables endpoints and replaces them with a single /stix endpoint. The /stix endpoint returns the threat feed as STIX indicators.
- Add `sightings` TAXII collection to represent IP addresses observed interacting with Deceptifeed as STIX sightings
- Rename collection aliases `deceptifeed-indicators` and `deceptifeed-observables` to `indicators` and `observables`.
- Add `convertToSightings` method for converting the threat feed to STIX sightings.
- Set a confidence score of 100 to STIX indicators.
- Rename label `honeypot` to `honeypot-interaction` on STIX indicators.
This change sets a maximum threat score of 999,999,999. Previously, the maximum threat score was the max signed int value (9,223,372,036,854,775,807 on 64-bit systems).
This commit ensures the `User-Agent` value is removed from the HTTP request headers when logging the request. `User-Agent` is logged in the `event_details` array, so it should not duplicate the value in the `headers` array when logging. This commit fixes the issue introduced in commit 12ada38faa which normalised header names to lowercase.
This change adds a default landing page that is served when accessing the threat feed root URL. The landing page provides information about accessing the threat feed.
Move the template from handleHTML to the templates directory and embed using embed.FS.
Several changes to HTTP routes:
- Delete `/csv/ips` route and handler.
- Delete `/json/ips` route and handler.
- Rename `/stix2` route to `/stix/indicators`.
- Rename `/stix2/ips` route to `/stix/observables`.
- Change default route `/` to return 404 Not Found.
- Move plain text feed to `/feed` and `/plain` routes.
- Add new route `/html` and associated handler for serving the threat feed as a web page for viewing in a browser.
- Revise sorting methods to apply a secondary sort by IP adddress when the values for the requested sort method are equal.
- Add a `feedEntry` struct to represent threat feed data as it is being served to clients. The threat feed is now served as a slice of feedEntries. Previously, the threat feed was built using a slice of net.IPs. The new struct avoids additional map lookups when formatting the feed and faster sorting operations.
- Rename `IoC` struct to `IOC` and make fields non-exportable.
- Revise HTTP handlers to format/encode the feed results using the new structs.
- Add `sortDirection` type to represent sort directions.
- Add sortDirection argument to applySort function.
- Add `direction` HTTP query parameter for setting the sort direction option.
- Add `sort` query parameter to sort the threat feed by IP (default), last seen date, added date, or by threat score.
- Add `last_seen_hours` query parameter to have the threat feed only return IPs that have been seen within the specified number of hours.
Examples:
`http://127.0.0.1:9000/json?sort=added`
`http://127.0.0.1:9000/?sort=ip&last_seen_hours=24`
- Add parseParams function to parse HTTP query parameters.
- Switch from functional options to an options struct for prepareFeed.
- Define a `feed` type for []net.IP.
- Add `applySort` method for sorting the feed.
- Change `convertToObservables` and `convertToIndicators` from functions to methods.
- Optimize exclude list processing.
- parseExcludeList now determines and tracks whether each entry is a single IP or a CIDR block and now returns both a map of single IPs and a slice of CIDR blocks.
- prepareFeed now calls parseExcludeList prior to iterating over iocData.
- Delete the filterIPs function. All filtering logic is applied while iterating over iocData rather than afterwards.
