Squid 4.0.25 release notes

Squid Developers


This document contains the release notes for version 4 of Squid. Squid is a WWW Cache application developed by the National Laboratory for Applied Network Research and members of the Web Caching community.

1. Notice

2. Major new features since Squid-3.5

3. Changes to squid.conf since Squid-3.5

4. Changes to ./configure options since Squid-3.5

5. Regressions since Squid-2.7

6. Copyright


1. Notice

The Squid Team are pleased to announce the release of Squid-4.0.25 for testing.

This new release is available for download from http://www.squid-cache.org/Versions/v4/ or the mirrors.

While this release is not deemed ready for production use, we believe it is ready for wider testing by the community.

We welcome feedback and bug reports. If you find a bug, please see http://wiki.squid-cache.org/SquidFaq/BugReporting for how to submit a report with a stack trace.

1.1 Known issues

Although this release is deemed good enough for use in many setups, please note the existence of open bugs against Squid-4.

This release adds a dependency on C++11 support in any compiler used to build Squid. As a result older C++03 -only and most C++0x compilers will no longer build successfully. GCC 4.9+ and Clang 3.5+ are known to have working C++11 support and are usable. GCC-4.8 will also build for now despite lack of full C++11 support, but some future features may not be available.

This release does not support LibreSSL. Due to a bug in the way LibreSSL uses the OpenSSL version macro some changes necessary to support OpenSSL 1.1 prevent building with LibreSSL.

1.2 Changes since earlier releases of Squid-4

The Squid-4 change history can be viewed here.

2. Major new features since Squid-3.5

Squid 4 represents a new feature release above 3.5.

The most important of these new features are:

Most user-facing changes are reflected in squid.conf (see below).

2.1 Configurable helper queue size

The new queue-size=N option to helpers configuration, allows users to configure the maximum number of queued requests to busy helpers.

2.2 Helper concurrency channels changes

helper-mux.pl we have been distributing for the past few years to encourage use of concurrency is no longer compatible with Squid. If used it will spawn up to 2^64 helpers and DoS the Squid server.

Helpers utilizing arrays to handle fixed amounts of concurrency channels MUST be re-written to use queues and capable of handling a 64-bit int as index or they will be vulnerable to buffer overrun and arbitrary memory accesses.

32-bit helpers need re-writing to handle the concurrency channel ID as a 64-bit integer value. If not updated they will cause proxies to return unexpected results or timeout once crossing the 32-bit wrap boundary. Leading to undefined behaviour in the client HTTP traffic.

2.3 SSL support removal

Details in RFC 6176 and RFC 7568

SSLv2 is not fit for purpose. Squid no longer supports being configured with any settings regarding this protocol. That includes settings manually disabling its use since it is now forced to disable by default. Also settings enabling various client/server workarounds specific to SSLv2 are removed.

SSLv3 is not fit for purpose. Squid still accepts configuration, but use is deprecated and will be removed entirely in a future version. Squid default behavour is to follow the TLS built in negotiation mechanism which prefers the latest TLS version. But also to accept downgrades to SSLv3. Use tls-options=NO_SSLv3 to disable SSLv3 support completely.

A new option tls-min-version=1.N is added in place of sslversion= to configure the minimum version the TLS negotiation will allow to be used when an old TLS version is requested by the remote endpoint.

The system Trusted CAs are no longer used by default when verifying client certificates. The cafile= option should be used instead to load the specific CA which signed acceptible client certificates explicitly, even if that CA is one of the system Trusted CAs. The tls-default-ca option can be used to restore the old behaviour explicitly if needed.

2.4 Helper Binary Changes

The basic_msnt_multi_domain_auth helper has been removed. The basic_smb_lm_auth helper performs the same actions without extra Perl and Samba dependencies.

The cert_valid.pl testing helper has been renamed to security_fake_certverify, reflecting the Squid helper naming schema and that it does not actually perform any certificate checks.

The security_fake_certverify helper is also now built and installed by default. It is written in Perl so does not require OpenSSL dependencies for installation. But does use the Perl Crypt::OpenSSL::X509 module for execution. Building the helper can be controlled using the --enable-security-cert-validators="fake" option.

The ssl_crtd helper has been renamed to security_file_certgen and is now built and installed by default whenever OpenSSL support is enabled. Building the helper can be controlled using the --enable-security-cert-generators="file" option. NOTE: The --enable-ssl-crtd option is still required to enable the sslcrtd_program helper interface within Squid that uses the helper.

The ntlm_smb_lm_auth helper is now built using --enable-auth-ntlm="SMB_LM". Notice the upper case where it was previously a (wrongly) lower cased acronym.

2.5 Secure ICAP

ICAP services can now be used over TLS connections.

To mark an ICAP service as secure, use an icaps:// service URI scheme when listing your service via an icap_service directive. The industry is using a Secure ICAP term, and Squid follows that convention, but icaps seems more appropriate for a scheme name.

Squid uses port 11344 for Secure ICAP by default, following another popular proxy convention. The old 1344 default for plain ICAP ports has not changed.

2.6 Improved SMP support

Use of C++11 atomic operations instead of GNU atomics allows a wider range of operating systems and compilers to build Squid SMP and multi-process features. However this does require a C++11 or C++0x compiler with a recent version of the C++ standard library.

IpcIo and Mmapped disk I/O modules are now auto-detected properly which enables Rock storage on more systems by default than previously.

2.7 Improved process management

Squid is traditionally refered to as a daemon. But is actually a combination of daemon and daemon manager processes. This has caused significant problems integrating it with other third-party daemon managers.

The Squid process which places its PID into the squid.pid file has always been the process to which control signals are sent. The manager process is now taking on signal handling instead of the main daemon process. Enabling integration with daemon managers such as Upstart or systemd which assume the process they initiated is the daemon with a PID to control.

The squid binary now has a new --foreground command line option, which (only) prevents daemonizing the master process. Unlike the old -N option, --foreground supports SMP workers and multi-process features. --foreground is particularly useful for use with -z (disk cache structures creation), as it allows the caller to wait until Squid has finished.

The squid binary now accepts a --kid command line option which informs the process which role it is to take on. This aids with debugging SMP issues with specific process types and resolves some SMP forking issues.

2.8 Initial GnuTLS support

Squid can now be built to use GnuTLS in place of OpenSSL for the core features of receiving TLS connections from clients and making TLS connections to servers. The GnuTLS support is still very much experimental and should be tested before use.

SSL-Bump and certificate generation features are not yet supported by GnuTLS builds. Nor are many other less commonly used Squid TLS/SSL features.

squid.conf directives and configuration options which have undergone name changes from 'ssl' to 'tls' prefix in Squid-4 have GnuTLS support, unless explicitly stated otherwise.

Advanced configuration with specific selection of ciphers and similar settings should still work, but needs the GnuTLS Priority Strings instead of the OpenSSL options when using GnuTLS.

2.9 ESI Custom Parser removal

The Squid custom ESI (Edge Side Includes) parser has been removed in favour of better supported and maintained third-party libraries. At least one of libxml2 or libexpat is now mandatory to build support for the ESI response processor.

3. Changes to squid.conf since Squid-3.5

There have been changes to Squid's configuration file since Squid-3.5.

This section gives a thorough account of those changes in three categories:

3.1 New tags

collapsed_forwarding_shared_entries_limit

New directive to limit the size of a table used for sharing information about collapsible entries among SMP workers.

force_request_body_continuation

New directive to control Squid behaviour on the client connection when receiving an HTTP request with an Expect:100-continue header.

hopeless_kid_revival_delay

New directive to set a cool-down delay reviving a child process if the process is encountering frequent deaths.

on_unsupported_protocol

New directive to set the action performed when encountering strange protocol requests at the beginning of an accepted TCP connection.

pconn_lifetime

New directive to limit the lifetime of persistent connections.

reply_header_add

New directive to add header fields to outgoing HTTP responses to the client.

request_start_timeout

New directive controlling how long Squid waits for the first request bytes to arrive after initial connection establishment by a client.

server_pconn_for_nonretriable

New directive to provide fine-grained control over persistent connection reuse when forwarding HTTP requests that Squid cannot retry. It is useful in environments where opening new connections is very expensive and race conditions associated with persistent connections are very rare and/or only cause minor problems.

shared_memory_locking

New directive to ensure shared memory is all available immediately on startup. Protects against SIGBUS errors, but delays startup.

tls_outgoing_options

New directive to define TLS security context options for outgoing connections. For example to HTTPS servers.

url_rewrite_timeout

Squid times active requests to redirector. This option sets the timeout value and the Squid reaction to a timed out request.

3.2 Changes to existing tags

access_log

TCP accept(2) errors logged with URI error:accept-client-connection.

Unused connections received in http_port or https_port or transactions terminated before reading[parsing] request headers logged with URI error:transaction-end-before-headers.

New option rotate= to control the number of log file rotations to make when -k rotate command is received. Default is to obey the logfile_rotate directive.

acl

New -m flag for note ACL to match substrings.

New clientside_mark type for matching Netfilter CONNMARK on the client TCP connection.

New connections_encrypted type for matching transactions where all HTTP messages were received over TLS transport connections, including messages received from ICAP servers.

New has type for matching whether or not Squid is able to provide certain sets of transaction state. For example HTTP reply headers.

New transaction_initiator type for detecting various unusual transactions.

New --consensus, --client-requested and --server-provided flags for the ssl::server_name type to control which server name to match against.

auth_param

New parameter queue-size= to set the maximum number of queued requests.

New parameter on-persistent-overload= to set the action taken when the helper queue is overloaded.

cache_peer

New option auth-no-keytab to let GSSAPI implementation determine which Kerberos credentials to use, instead of specifying a keytab.

Replaced option ssl with tls. Use of any tls- prefixed options implies tls is enabled.

New option tls-min-version=1.N to set minimum TLS version allowed.

New option tls-default-ca replaces sslflags=NO_DEFAULT_CA

New option tls-no-npn to disable sending TLS NPN extension.

All ssloptions= values for SSLv2 configuration or disabling have been removed.

Removed sslversion= option. Use tls-options= instead.

Manual squid.conf update may be required on upgrade.

Replaced option sslcafile= with tls-cafile= which takes multiple entries.

deny_info

New format macro %O to expand the message= value supplied by external ACL helpers.

ecap_service

New connection-encryption= option to determine ICAP service effect on connections_encrypted ACL.

esi_parser

Removed custom parser option.

Changed default to auto-detect available parsers instead of custom.

external_acl_type

New parameter queue-size= to set the maximum number of queued requests.

New parameter on-persistent-overload= to set the action taken when the helper queue is overloaded.

Format field updated to accept any logformat %macro code.

http_port

New option tls-min-version=1.N to set minimum TLS version allowed.

New option tls-default-ca replaces sslflags=NO_DEFAULT_CA, the default is also changed to OFF.

New option tls-no-npn to disable sending TLS NPN extension.

All option= values for SSLv2 configuration or disabling have been removed.

Removed version= option. Use tls-options= instead.

Manual squid.conf update may be required on upgrade.

Replaced cafile= with tls-cafile= which takes multiple entries.

Changed default value of generate-host-certificates to ON.

https_port

New option tls-min-version=1.N to set minimum TLS version allowed.

New option tls-default-ca replaces sslflags=NO_DEFAULT_CA, the default is also changed to OFF.

New option tls-no-npn to disable sending TLS NPN extension.

All options= values for SSLv2 configuration or disabling have been removed.

Removed version= option. Use tls-options= instead.

Manual squid.conf update may be required on upgrade.

Replaced cafile= with tls-cafile= which takes multiple entries.

Changed default value of generate-host-certificates to ON.

icap_service

New scheme icaps:// to enable TLS/SSL connections to Secure ICAP servers on port 11344.

New connection-encryption= option to determine ICAP service effect on connections_encrypted ACL.

New tls-cert= option to set TLS client certificate to use.

New tls-key= option to set TLS private key matching the client certificate used.

New tls-min-version=1.N option to set minimum TLS version allowed on server connections.

New tls-options= option to set OpenSSL library parameters.

New tls-flags= option to set flags modifying Squid TLS operations.

New tls-cipher= option to set a list of ciphers permitted.

New tls-cafile= option to set a file with additional CA certificate(s) to verify the server certificate.

New tls-capath= option to set a directory with additional CA certificate(s) to verify the server certificate.

New tls-crlfile= option to set a file with a CRL to verify the server certificate.

New tls-default-ca option to use the system Trusted CAs to verify the server certificate.

New tls-domain= option to verify the server certificate domain.

logfile_daemon

Now only requires that helper binary exists when daemon: log module is actually being used.

logformat

New quoting modifier to produce \-escaped output.

New code %ssl::<cert_errors to display server X.509 certificate errors.

New code %ssl::<cert_issuer to display Issuer field of the received server X.509 certificate.

New code %ssl::<cert_subject to display Subject field of the received server X.509 certificate.

New code %ssl::>negotiated_version to display negotiated TLS version of the client connection.

New code %ssl::<negotiated_version to display negotiated TLS version of the last server or peer connection.

New code %ssl::>received_hello_version to display the TLS version of the Hello message received from TLS client.

New code %ssl::<received_hello_version to display the TLS version of the Hello message received from TLS server.

New code %ssl::>received_supported_version to display the maximum TLS version supported by the TLS client.

New code %ssl::<received_supported_version to display the maximum TLS version supported by the TLS server.

New code %ssl::>negotiated_cipher to display the negotiated cipher of the client connection.

New code %ssl::<negotiated_cipher to display the negotiated cipher of the last server or peer connection.

Fixed %<Hs, %<pt and %<tt codes for received CONNECT errors.

Improved %<bs logging on forwarding retries.

Improved %<Hs, %<pt, %<tt, %<bs logging on SslBump errors.

pid_filename

Default value now based on squid -n command line parameter.

This directive is no longer mandatory to edit for multi-instance/tenant Squid installations.

refresh_pattern

Removed option ignore-auth. Its commonly desired behaviour is performed by default with correct HTTP/1.1 revalidation.

Removed option ignore-must-revalidate. Other more HTTP compliant directives (cache, store_miss) can be used to prevent objects from caching.

sslcrtd_children

New parameter queue-size= to set the maximum number of queued requests.

New parameter on-persistent-overload= to set the action taken when the helper queue is overloaded.

sslcrtvalidator_children

New parameter queue-size= to set the maximum number of queued requests.

New parameter on-persistent-overload= to set the action taken when the helper queue is overloaded.

store_id_children

New parameter queue-size= to set the maximum number of queued requests.

New parameter on-persistent-overload= to set the action taken when the helper queue is overloaded.

url_rewrite_children

New parameter queue-size= to set the maximum number of queued requests.

New parameter on-persistent-overload= to set the action taken when the helper queue is overloaded.

3.3 Removed tags

cache_peer_domain

Superceded by cache_peer_access. Use dstdomain ACL in the access control list to restrict domains requested.

ie_refresh

Removed. MSIE 3.x, 4.x, 5.0 and 5.01 are no longer popular browsers.

sslproxy_cafile

Replaced by tls_outgoing_options cafile=. Which now takes multiple entries.

sslproxy_capath

Replaced by tls_outgoing_options capath=.

sslproxy_cipher

Replaced by tls_outgoing_options cipher=.

sslproxy_client_certificate

Replaced by tls_outgoing_options cert=.

sslproxy_client_key

Replaced by tls_outgoing_options key=.

sslproxy_flags

Replaced by tls_outgoing_options flags=.

sslproxy_options

Replaced by tls_outgoing_options options=.

All values for SSLv2 configuration or disabling have been removed.

Manual squid.conf update may be required on upgrade.

sslproxy_version

Replaced by tls_outgoing_options options=.

All values for SSLv2 configuration or disabling have been removed.

Manual squid.conf update may be required on upgrade.

4. Changes to ./configure options since Squid-3.5

There have been some changes to Squid's build configuration since Squid-3.5.

This section gives an account of those changes in three categories:

4.1 New options

--enable-security-cert-generators

New option to control which TLS/SSL dynamic certificate generator helpers are built and installed.

Helper ssl_crtd has been renamed to security_file_certgen and built with module name file. Requires --with-openssl.

--enable-security-cert-validators

New option to control which TLS/SSL certificate validation helpers are built and installed.

One fake helper that does not actually perform any certificate checks is provided for testing and as an example for writing custom helpers.

--without-cppunit

The cppunit testing framework is auto-detected and used when available. This option can be used to disable it explicitly.

4.2 Changes to existing options

--enable-auth-basic

The MSNT-multi-domain helper has been removed.

The SMB LanMan helper SMB_LM is no longer built by default. It needs to be explicitly listed to be built.

--enable-auth-ntlm

The SMB LanMan helper is now built using SMB_LM (was lower case smb_lm).

The SMB LanMan helper SMB_LM is no longer built by default. It needs to be explicitly listed to be built.

--enable-diskio

Auto-detection of SMP related modules has been fixed to actually auto-detect them without configuring the module list manually.

--enable-esi

Custom ESI parser has been removed. Libxml2 or libexpat is now required to enable ESI processing.

4.3 Removed options

--with-cppunit-basedir

Replaced by --with-cppunit=PATH. Please prefer the default auto-detection though.

XSTD_USE_LIBLTDL

Removed. Use --with-included-ltdl instead.

5. Regressions since Squid-2.7

Some squid.conf options which were available in Squid-2.7 are not yet available in Squid-4

If you need something to do then porting one of these from Squid-2 to Squid-3 is most welcome.

5.1 Missing squid.conf options available in Squid-2.7

broken_vary_encoding

Not yet ported from 2.6

cache_peer

monitorinterval= not yet ported from 2.6

monitorsize= not yet ported from 2.6

monitortimeout= not yet ported from 2.6

monitorurl= not yet ported from 2.6

cache_vary

Not yet ported from 2.6

error_map

Not yet ported from 2.6

external_refresh_check

Not yet ported from 2.7

location_rewrite_access

Not yet ported from 2.6

location_rewrite_children

Not yet ported from 2.6

location_rewrite_concurrency

Not yet ported from 2.6

location_rewrite_program

Not yet ported from 2.6

refresh_pattern

stale-while-revalidate= not yet ported from 2.7

ignore-stale-while-revalidate= not yet ported from 2.7

negative-ttl= not yet ported from 2.7

refresh_stale_hit

Not yet ported from 2.7

update_headers

Not yet ported from 2.7

6. Copyright

Copyright (C) 1996-2018 The Squid Software Foundation and contributors

Squid software is distributed under GPLv2+ license and includes contributions from numerous individuals and organizations. Please see the COPYING and CONTRIBUTORS files for details.