23.8.11 Altinity Stable

Changes compared to upstream's v23.8.11.28-lts

Bug fixes

• Fixes partition pruning for extra columns in a set. (ClickHouse#55172 by @amosbird via #342)
• Fixed rare bug when we produced invalid read ranges for queries with FINAL. Resulted in Cannot read out of marks range exception. (ClickHouse#54934 by @nickitat via #343)
• Fix incorrect free space accounting for least_used JBOD policy (ClickHouse#56030 by @azat via #344)
• Fix ALTER COLUMN with ALIAS that previously threw the NO_SUCH_COLUMN_IN_TABLE exception. (ClickHouse#57395 by @evillique via #345)
• Disable system.kafka_consumers by default (due to possible live memory leak) (ClickHouse#57822 by @azat via #346)

Performance improvements

• Fixed filtering by IN(...) condition for Merge table engine. (ClickHouse#54905 by @nickitat via #341)

Build/Testing/Packaging Improvement

• Make builds possible on Altinity's infrastructure
• clickhouse-regression test suite (#338 by @MyroTk)
• Changed "version is not official" log message on failure (#369 by @Enmk)
• Fix kerberized_hadoop docker image build issue (#394 by @ilejn)

Improvement

• Support S3 access through AWS Private Link Interface endpoints. (ClickHouse#62208 by @arthurpassos via #389)
• Fixed accounting of memory allocated before attaching thread to a query or a user (ClickHouse#56089 by @nickitat via #388)

upstream ClickHouse changes between v23.8.11.28-lts (31879d2) and v23.8.8.20-lts (5e012a0)

Improvement

• Output valid JSON/XML on excetpion during HTTP query execution. Add setting http_write_exception_in_output_format to enable/disable this behaviour (enabled by default). #52853
• Fix transfer query to MySQL compatible query. Fixes #57253. Fixes #52654. Fixes #56729. #56456
• Fetching a part waits when that part is fully committed on remote replica. It is better not send part in PreActive state. In case of zero copy this is mandatory restriction. #56808
• Handle sigabrt case when getting PostgreSQl table structure with empty array. #57618
• Add SYSTEM JEMALLOC PURGE for purging unused jemalloc pages, SYSTEM JEMALLOC [ ENABLE | DISABLE | FLUSH ] PROFILE for controlling jemalloc profile if the profiler is enabled. Add jemalloc-related 4LW command in Keeper: jmst for dumping jemalloc stats, jmfp, jmep, jmdp for controlling jemalloc profile if the profiler is enabled. #58665
• Copy S3 file GCP fallback to buffer copy in case GCP returned Internal Error with GATEWAY_TIMEOUT HTTP error code. #60164
• Update tzdata to 2024a. #60768

Build/Testing/Packaging Improvement

• Fix issue caught in docker-library/official-images#15846. #57571
• If you want to run initdb scripts every time when ClickHouse container is starting you shoud initialize environment varible CLICKHOUSE_ALWAYS_RUN_INITDB_SCRIPTS. #59808

Bug Fix (user-visible misbehavior in an official stable release)

• Flatten only true Nested type if flatten_nested=1, not all Array(Tuple) #56132
• Fix ALTER COLUMN with ALIAS #56493
• Prevent incompatible ALTER of projection columns #56948
• Fix segfault after ALTER UPDATE with Nullable MATERIALIZED column #57147
• Fix incorrect JOIN plan optimization with partially materialized normal projection #57196
• Fix ReadonlyReplica metric for all cases #57267
• Fix working with read buffers in StreamingFormatExecutor #57438
• bugfix: correctly parse SYSTEM STOP LISTEN TCP SECURE #57483
• Ignore ON CLUSTER clause in grant/revoke queries for management of replicated access entities. #57538
• Disable system.kafka_consumers by default (due to possible live memory leak) #57822
• Fix invalid memory access in BLAKE3 (Rust) #57876
• Normalize function names in CREATE INDEX #57906
• Fix invalid preprocessing on Keeper #58069
• Fix Integer overflow in Poco::UTF32Encoding #58073
• Remove parallel parsing for JSONCompactEachRow #58181
• Fix parallel parsing for JSONCompactEachRow #58250
• Background merges correctly use temporary data storage in the cache #57275
• MergeTree mutations reuse source part index granularity #57352
• Fix double destroy call on exception throw in addBatchLookupTable8 #58745
• Fix JSONExtract function for LowCardinality(Nullable) columns #58808
• Fix: LIMIT BY and LIMIT in distributed query #59153
• Fix translate() with FixedString input #59356
• Fix error "Read beyond last offset" for AsynchronousBoundedReadBuffer #59630
• Fix query start time on non initial queries #59662
• Fix leftPad / rightPad function with FixedString input #59739
• rabbitmq: fix having neither acked nor nacked messages #59775
• Fix cosineDistance crash with Nullable #60150
• Fix buffer overflow in CompressionCodecMultiple #60731
• Remove nonsense from SQL/JSON #60738
• Fix crash in arrayEnumerateRanked #60764
• Fix crash when using input() in INSERT SELECT JOIN #60765
• Remove recursion when reading from S3 #60849

NO CL ENTRY

• Update PeekableWriteBuffer.cpp. #57701
• Use the current branch test-utils to build cctools. #61276

NOT FOR CHANGELOG / INSIGNIFICANT

• Pin alpine version of integration tests helper container #57669
• Remove heavy rust stable toolchain #57905
• Fix docker image for integration tests (fixes CI) #57952
• Fix rare race in external sort/aggregation with temporary data in cache #58013
• Fix possible race in ManyAggregatedData dtor. #58624
• Fix 02720_row_policy_column_with_dots #59453
• Pin python dependencies in stateless tests #59663
• Make ZooKeeper actually sequentialy consistent #59735
• Remove broken test while we fix it #60547
• Cancel PipelineExecutor properly in case of exception in spawnThreads #57104
• Detect io_uring in tests #60373
• Cancel PipelineExecutor properly in case of exception in spawnThreads #60499

Packages

Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/ as either .deb, .rpm, or .tgz

Docker images

Available for both AMD64 and Aarch64: altinity/clickhouse-server/23.8.11.29.altinitystable