Go agent release notes


v0.9.0 - 19 February 2020

GitHub Release

This new major version says farewell to the beta and adds SQL-injection run time protection thanks the first building blocks of RASP for the Go language! Thank you to everyone that helped us in this wonderful and amazing journey ;-)

The Go agent has been protecting production servers for more than a year now and we reached the point where we are confident enough about its deployment, setup, but also its internals and specific integrations with the Go language and runtime.

We are getting closer to the fully-featured agent v1.0 as we will now be able to fully add support for every RASP protection Sqreen supports.

Breaking Changes

Because we now want a stable public API, find below the breaking changes:

  • The former separate agent package github.com/sqreen/go-agent/agent that was asked to import in order to start the agent is no longer required nor available. This is now performed by the middleware functions we provide in order to avoid the most common setup mistake during the beta where only the agent was setup and no middleware function was set to protect the requests (and therefore nothing was happening).

  • SDK: the user identification SDK method Identify() has been updated to be simpler to use and less error-prone by now making it return a non-nil error when the request handler shouldn't continue any further serving the request. It happens when a user security response has matched the identified user. This replaces the former separate SDK method MatchSecurityResponse(). New usage example:

    sqUser := sq.ForUser(sdk.EventUserIdentifiersMap{"uid": "unique user id"})
    if err := sqUser.Identify(); err != nil {
      return
    }
    

  • The agent no longer starts if the program wasn't instrumented using the instrumentation tool. See docs.sqreen.com/go/installation for details on how to install and use the tool. Note that the program is not aborted - only the agent is disabled.

  • Dropping gRPC support: the beta support for gRPC was experimental and was in the end too limited by Sqreen's focus on the HTTP protocol. Most of our protections are indeed designed for HTTP and couldn't be applied at the gRPC protocol level. We are therefore removing it until we can provide a correct experience for such HTTP-based protocol.
    Please contact us if you need any further information or if you are interested in helping us building it (support@sqreen.com).

New Features

  • SQL-injection RASP protection: when enabled on Sqreen's dashboard, the database/sql Go package gets automatically protected against SQL injections. SQL queries go through our SQL-injection detection which will abort the SQL function call and corresponding HTTP request when an attack is detected.
    Note that special care was taken to properly intergrate with Go error-handling principles: when a SQL query gets blocked, the HTTP request context is canceled and a non-nil error is returned by the database/sql function call in order to fall into the existing error-handling flow. For example:

    // The following query can be injected. An error is returned when the SQL
    // query was blocked.
    rows, err := db.QueryContext(ctx, "select id, name from users where id=" + unsafe)
    if err != nil {
      return err
    }
    
    Read more about Go integration details at https://docs.sqreen.com/go/integration.

  • Dashboard diagnostic messages: major setup issues are now also reported through Sqreen's dashboard page of running hosts to get notified about some downgraded states of the agent, such as:

  • The Go program is not instrumented so the agent didn't start.
  • The In-App WAF wasn't compiled (eg. CGO disabled) so it is unavailable and disabled.
  • The program dependencies couldn't be retrieved because the program was not compiled as a Go module. This is also shown by the dashboard when the list of dependencies is empty.

v0.1.0-beta.10 - 24 January 2020

GitHub Release

Breaking Change

  • (#89) Go instrumentation: Sqreen's dynamic configuration of the protection your Go programs is made possible at run time thanks to Go instrumentation. It is a building block of the upcoming run time self-protection (aka RASP) and it is safely performed at compilation time by an instrumentation tool that seamlessly integrates with the Go toolchain. To begin with, only a specific set of Go packages are instrumented: the agent and database/sql (to prepare the upcoming SQL injection protection).

Please, find out how to install and use the tool on the new agent installation documentation available at https://docs.sqreen.com/go/installation/.

New Features

  • (#90) The SDK now imports the agent package to no longer have to import it in the main package. The SDK is indeed mandatory when setting up Sqreen for Go, making it the best place to import the agent.

  • (#91) The program dependencies are now sent to Sqreen to perform dependency analysis (outdated, vulnerable, etc.). They are only available when the Go program you compile is a Go module. Sqreen's dashboard Dependency page will be made available soon.

Fix

  • (#92) Vendoring using go mod vendor could lead to compilation errors due to missing files.

v0.1.0-beta.9 - 19 December 2019

GitHub Release

New Features

  • Request parameters such as query or post parameters are now added in the attack events and shown in the attack logs and in the event explorer pages of our dashboard. (#84)

  • PII scrubbing is now performed on every event sent to Sqreen, as documented here. (#86)

Fixes

  • Add PII scrubbing to the WAF logs that may include data from the request. (#87)

Internal Changes

  • The In-App WAF has been intensively optimized so that large requests can no longer impact its execution time. (#83)

v0.1.0-beta.8 - 15 October 2019

GitHub Release

Internal Changes

  • In-App WAF:
    • Dynamically set the WAF timeout (#79).
    • Ignore WAF timeout errors and add more context when reporting an error (#80).
    • Update the libsqreen to v0.4.0 to add support for the @pm operator.

v0.1.0-beta.7 - 26 September 2019

GitHub Release

Breaking Changes

  • CGO bindings are now involved in the compilation of the agent and will require the gcc compiler and the C library headers. Note that compiling the agent without CGO (CGO_ENABLED=0) is still possible but would disable some agent features; in this case the new In-App WAF feature described below.

New Feature

  • Add support for the In-App WAF: an out-of-the-box Web-Application Firewall leveraging the full application context, that is fail-safe, has limited false positives and won’t require heavy fine-tuning. Only darwin/amd64 and linux/amd64 targets are supported so far. Any other target will get this feature disabled. More targets will be added in future versions. (#77)

Minor Change

  • Increase the internal timeout value of the HTTP client to Sqreen's backend in order to be more resilient to normal networking delays.

Fix

  • Fix a compilation error on 32-bit target architectures.

v0.1.0-beta.6 - 25 July 2019

GitHub Release

New Features

  • Fully-featured playbooks with the added ability into the agent to redirect the request to a given URL. (#72)

  • Configurable protection behaviour of the agent when blocking a request by either customizing the HTTP status code that is used for the blocking HTML page, or by redirecting to a given URL instead.
    Dashboard page: https://my.sqreen.com/application/goto/settings/global#protection-mode

  • HTTP response status code monitoring. (#75)
    Dashboard page: https://my.sqreen.com/application/goto/monitoring

  • Support for browser security headers protection modules allowing to enable various browser security options allowing to restrict modern browsers from running into some preventable vulnerabilities:

  • [Content Security Policy][csp] protection module allowing to prevent cross-site scripting attacks. (#74)
    Dashboard page: https://my.sqreen.com/application/goto/modules/csp

  • Security headers protection module allowing to protect against client-side vulnerabilities in the browser. (#73)
    Dashboard page: https://my.sqreen.com/application/goto/modules/headers

Minor Changes

  • Better agent configuration logs clearly stating where does the configuration come from (file in search path, enforced file or environment variables), along with the possibility to display the full settings using the debug log-level.

v0.1.0-beta.5 - 23 May 2019

GitHub Release

New Features

  • Middleware functions for gRPC over HTTP2 (called interceptors). Details on https://godoc.org/github.com/sqreen/go-agent/sdk/middleware/sqgrpc (#23)

  • IP whitelist: support to make the agent completely ignore requests whose IP addresses match a whitelist entry. Everything related to Sqreen, including events, will be ignored for whitelisted requests. (#69)

  • Agent failsafe modes: the agent cannot stop the Go app by catching every internal error. It either tries to restart it or stop it. Panic-ing and stopping the host app is no longer an option. (#67)

Minor Changes

  • Internal event batch improvements:

    • Increased batch buffer capacity from 60 to 6000 entries in order to be able to handle more events, sent by batches of 60 events per heartbeat.

    • Remove the batch bookkeeping goroutine and include its logic into the main event processing loop.

v0.1.0-beta.4 - 16 April 2019

This release adds the ability to block IP addresses or users in your Go web services by adding support for security automation according to your playbooks and their configured security responses.

Note that redirecting users or IP addresses is not supported yet.

GitHub Release

New features

Security automation

It's now possible to block IP addresses or users. When a playbook triggers, it notifies the agent and provides the batch of security responses. They are asynchronously stored into data structures optimized for fast lookup and low memory usage. Middleware functions can thus perform fast lookups to block requests in a few microseconds to exit request handlers as fast as possible. Learn more in our security automation documentation.

Blocking IP addresses

You don't need to make any changes to continue blocking IP addresses. We have updated our middleware functions to block requests whose IP addresses match a security response. Sqreen aborts the request with HTTP status code 500 and Sqreen's default HTML information page.

Blocking users

Blocking users by combining SDK methods Identify() and MatchSecurityResponse() to firstly associate a user to the current request, and secondly to check if it matches a security response. When a security response matches, stop the request handler and any related goroutines as soon as possible.

Usage example:

uid := sdk.EventUserIdentifiersMap{"uid": "my-uid"}
sqUser := sdk.FromContext(ctx).ForUser(uid)
sqUser.Identify()
if match, err := sqUser.MatchSecurityResponse(); match {
  // Return now to stop further handling the request and let Sqreen's
  // middleware apply the configured security response and abort the
  // request. The returned error may help aborting from sub-functions by
  // returning it to the callers when the Go error handling pattern is
  // used.
  return err
}

We strongly recommend creating a user-authentication middleware function to seamlessly integrate user-blocking in all your user-authenticated endpoints.

Fix

  • Escape the event type name to avoid JSON marshaling error. Note that this case could not happen in previous agent versions. (#52)

Minor change

  • Avoid repeating commands within the same command batch. (51)

v0.1.0-beta.3 - 22 March 2019

GitHub Release

New Features

  • Get the client IP address out of the HAProxy header X-Unique-Id using the new configuration variable ip_header_format.

  • New configuration option strip_http_referer/SQREEN_STRIP_HTTP_REFERER allowing to avoid sending the Referer HTTP header to the Sqreen backend when it contains sensitive data.

  • Ability to disable/enable the agent through the dashboard settings using the Sqreen status button.

Breaking Changes

  • Agent internals are now under a private Go package and can no longer be imported. Any sub-package under github.com/sqreen/go-agent/agent was not supposed to be imported and is now private to avoid future confusions.

Fixes

  • Remove duplicate User-Agent entry sent twice in the request record.

  • Fix IPv4 and IPv6 matching against private network definitions.

  • Remove useless empty request records mistakenly created while not carrying any SDK observation.

Minor Changes

  • Better memory management and footprint when the agent is disabled by removing globals. This will be also required to be able to cleanly restart the agent by self-managing the initializations.

v0.1.0-beta.2 - 14 February 2019

GitHub Release

New feature

  • Add a new Identify() SDK method allowing to explicitly associate a user to the current request. As soon as we add the support for the security reponses, it will allow to block users.

v0.1.0-beta.1 - 7 February 2019

GitHub Release

This version is a new major version towards the v0.1.0 as it proposes a new andstable SDK API, that now will only be updated upon user feedback. So please, share your impressions with us.

New Features

  • New web framework middleware supports:
  • Standard Go's net/http package.
  • Echo.

  • Multiple custom events can now be easily associated to a user using the user-scoped methods under ForUser(). For example, to send two custom events for a given user, do:

sqUser := sqreen.ForUser(uid)
sqUser.TrackEvent("my.event.one")
sqUser.TrackEvent("my.event.two")
  • The configuration file can now be stored into multiple locations, the current working directory or the executable one, or enforced using the new configuration environment variable SQREEN_CONFIG_FILE.

  • The custom client IP header configured in SCREEN_IP_HEADER is now also sent to Sqreen so that it can better understand what IP headers were considered by the agent to determine what is the actual client IP address.

Breaking Changes

  • Stable SDK API of "Sqreen for Go":

  • Avoid name conflicts with framework packages by prefixing Sqreen's middleware packages with sq. For example, gin becomes sqgin.

  • Cleaner Go documentation now entirely included in the SDK and middleware packages Go documentations. So no more need to go inside the agent documentation to know more on some SDK methods, it is now all documented in the same place, with lot of examples.

  • Clearer SDK API: The flow of security events that can send to Sqreen is now well-defined by a tree of SDK methods that can only be used the right way.

    • The SDK handle getter function name is renamed from GetHTTPRequestContext() into a simpler FromContext().

    • User-related SDK methods are now provided by ForUser(), for example: sqreen.TrackAuth(true, uid) becomes sqreen.ForUser(uid).TrackAuthSuccess().

v0.1.0-alpha.5 - 24 January 2019

GitHub Release

New features

  • SDK:

    • Ability to associate a user to a single event using WithUserIdentifier().
    • Track user creations using TrackSignup().
    • Track user authentications using TrackAuth().
  • Agent:

    • Proxy configuration: take into account {HTTPS,HTTP,NO}_PROXY environment variables (and their lowercase alternatives).
    • Token: organization token support.

Fixes

  • agent/config: avoid conflicts with global viper configs.
  • sdk: better documentation with examples.