This change adds a /config handler for displaying the Deceptifeed configuration. An icon is added to the nav bar for accessing the page.
- Add config.html template for displaying the Deceptifeed configuration.
- Add config.html supporting styles to style.css.
- Add /config http handler to render the config.html template.
- Add icon to nav bar for accessing /config page.
This change removes the `CustomThreatsPath` setting from the threat feed configuration. The default configuration files are updated with this setting removed.
This change removes the 'threat score' feature which allowed users to configure each honeypot server with a variable 'score' when updating the threat feed.
It is replaced with a fixed observation count that is incremented by 1 for each honeypot interaction.
The field `threat_score` has been replaced with `observations` in all API call parameters and threat feed data.
The `threat_score` field in the CSV file has been renamed to `observations`. Existing threat feed CSV files will be automatically updated on the next threat feed save.
- Add a horizontal navigation bar to the top of all management pages.
- Tweaks to colors and styling of management pages.
- Rename the endpoint '/threatfeed' to '/webfeed'.
- Rename the file 'htmlfeed.html' to 'webfeed.html'.
- Remove the `/feed` endpoint, as this is already handled by `/plain`.
- Rename `/html` endpoint to `/threatfeed`.
- Add a `/docs` endpoint for displaying information on accessing the threat feed.
- Make the Deceptifeed logo smaller on HTML pages.
- Revise the layout of the initial homepage. Endpoint docs are now moved and linked to /docs.
- Remove `/html` from the list of threat feed endpoints in the docs, as it doesn't belong here (there is a link to the html threat feed on the home page).
This change updates the threat feed server so that the entire application config is passed in to the threat feed server. This allows the threat feed server to access its own configuration as well as the settings for each honeypot server and any other application-wide settings.
This change removes the /stix/indicators and /stix/observables endpoints and replaces them with a single /stix endpoint. The /stix endpoint returns the threat feed as STIX indicators.
- Add `sightings` TAXII collection to represent IP addresses observed interacting with Deceptifeed as STIX sightings
- Rename collection aliases `deceptifeed-indicators` and `deceptifeed-observables` to `indicators` and `observables`.
- Add `convertToSightings` method for converting the threat feed to STIX sightings.
- Set a confidence score of 100 to STIX indicators.
- Rename label `honeypot` to `honeypot-interaction` on STIX indicators.
This change adds a default landing page that is served when accessing the threat feed root URL. The landing page provides information about accessing the threat feed.
Move the template from handleHTML to the templates directory and embed using embed.FS.
Several changes to HTTP routes:
- Delete `/csv/ips` route and handler.
- Delete `/json/ips` route and handler.
- Rename `/stix2` route to `/stix/indicators`.
- Rename `/stix2/ips` route to `/stix/observables`.
- Change default route `/` to return 404 Not Found.
- Move plain text feed to `/feed` and `/plain` routes.
- Add new route `/html` and associated handler for serving the threat feed as a web page for viewing in a browser.
- Revise sorting methods to apply a secondary sort by IP adddress when the values for the requested sort method are equal.
- Add a `feedEntry` struct to represent threat feed data as it is being served to clients. The threat feed is now served as a slice of feedEntries. Previously, the threat feed was built using a slice of net.IPs. The new struct avoids additional map lookups when formatting the feed and faster sorting operations.
- Rename `IoC` struct to `IOC` and make fields non-exportable.
- Revise HTTP handlers to format/encode the feed results using the new structs.
- Add `sortDirection` type to represent sort directions.
- Add sortDirection argument to applySort function.
- Add `direction` HTTP query parameter for setting the sort direction option.
- Add `sort` query parameter to sort the threat feed by IP (default), last seen date, added date, or by threat score.
- Add `last_seen_hours` query parameter to have the threat feed only return IPs that have been seen within the specified number of hours.
Examples:
`http://127.0.0.1:9000/json?sort=added`
`http://127.0.0.1:9000/?sort=ip&last_seen_hours=24`
- Add parseParams function to parse HTTP query parameters.
- Switch from functional options to an options struct for prepareFeed.
- Define a `feed` type for []net.IP.
- Add `applySort` method for sorting the feed.
- Change `convertToObservables` and `convertToIndicators` from functions to methods.
- Optimize exclude list processing.
- parseExcludeList now determines and tracks whether each entry is a single IP or a CIDR block and now returns both a map of single IPs and a slice of CIDR blocks.
- prepareFeed now calls parseExcludeList prior to iterating over iocData.
- Delete the filterIPs function. All filtering logic is applied while iterating over iocData rather than afterwards.
This change adds filtering and pagination when requesting objects from collections.
- Add `more` and `next` properties to the TAXII `envelope` resource.
- Add support for the `limit` URL query parameter to limit the maximum number of objects returned.
- Add full support for pagination. This includes:
- Sett the `X-TAXII-Date-Added-First` and `X-TAXII-Date-Added-Last` headers with the timestamps of the first and last objects from the results.
- Set the `more` property in the response to indicate whether there are more items remaining.
- Set the `next` property in the response with the next page number if there are more items remaining.
Other changes:
- Modiy the threat feed `last seen` sort to sort by IP when the dates are equal.
This change adds support for serving the threat feed over TAXII 2.1. The TAXII API is not yet fully implemented, but should work with most threat intelligence platforms.
Missing TAXII features:
- Mising `limit` query parameter.
- Missing `match` query parameter.
- Pagination is not supported.
- Missing `X-TAXII` headers.
- {api-root}/collections/{id}/manifest/
- {api-root}/collections/{id}/objects/{object-id}/
- {api-root}/collections/{id}/objects/{object-id}/versions/
- Add `internal\taxi\taxi.go` with struct definitions and helpers for TAXII resources.
- Move threat feed parsing functions from `handler.go` to `feed.go`.
- Add option functions for `prepareFeed` to allow sorting by last seen date and to filter by last seen date.
- Default timestamps to time.Now when initially loading feed data.
- Add new `stix` package for STIX-specific types and functions.
- Use `stix` package to decalre STIX types, rather than using local function declarations.
- Add function to return the Deceptifeed application represented as a STIX Identity object. Indicators and observables now reference this identity as the creator.
- Move `threatfeed\util.go` to `stix\uuid.go`.
- Move STIX conversions to dedicated functions to simplify HTTP handlers.
This change switches to using the STIX namespace for generating deterministic identifiers for STIX Domain Objects (SDOs). While the STIX specification states SDOs must not use the STIX namespace, some applications (OpenCTI) do so. This update improves compatibility with OpenCTI by adopting the same behavior.
This change adds support for serving the threat feed in a STIX 2.1 format using the `/stix2` and `/stix2/ips` routes.
- Add `handleSTIX2` function for the `/stix2` route. This returns the detailed threat feed structured as a STIX bundle containing *Indicator* objects for each IP address in the feed.
- Add `handleSTIX2Simple` handler for the `/stix2/ips` route. This returns the simplified threat feed structured as a STIX bundle with each IP address included as a STIX Cyber-observable Object (SCO).
- Add `util.go` with helper functions for creating STIX 2.1 identifiers. This includes a `newUUIDv5` function, a `newUUIDv4` function, and a `namespace` type.
- Rename database.go to data.go.
- Move data-related global vars from threatfeed.go to data.go.
- Split out functions from threatfeed.go into seperate files:
- Move HTTP server functions to server.go.
- Move HTTP handler functions to handler.go.
- Move HTTP middleware functions to middleware.go.
- Rename hasMapChanged to dataChanged.
