DuckDB 1.4.0 "Andium"

This release of DuckDB is named "Andium" after Anas Andium, a species of duck that lives in the Andes mountains in South America.

Please also refer to the announcement blog post: https://duckdb.org/2025/09/16/announcing-duckdb-140.html

What's Changed

• Python package devexp improvements by @evertlammerts in #17483
• change exception type to not be an internal exception by @samansmink in #17551
• Remove redundant code path in the ConflictManager by @taniabogatsch in #17562
• Add support for ToSqlString for union types by @wmTJc9IK0Q in #17513
• Update function descriptions and examples by @c-herrewijn in #17132
• Move query profiler's EndQuery after commit/rollback by @taniabogatsch in #17595
• fix extension troubleshooting link by @simon0191 in #17616
• C API tidying by @taniabogatsch in #17623
• bump DuckDB_jll to v1.3.0 by @c-herrewijn in #17677
• Add rowsort in generate_series test #43 by @jeewonhh in #17675
• [C API] Expose duckdb_scalar_function_bind_get_extra_info by @taniabogatsch in #17666
• Enable profiling output for all operator types by @taniabogatsch in #17665
• Output hashes in unittest and fix order by @niykko in #17664
• New Sorting Implementation by @lnkuiper in #17584
• Merge v1.3-ossivalis into main by @Mytherin in #17690
• Issue #17040: FILL Window Function by @hawkfish in #17686
• ClientBufferManager wrapper to access the client context in the buffer manager by @taniabogatsch in #17699
• Revert "set default for MAIN_BRANCH_VERSIONING to false" by @carlopi in #17708
• Sorting followup by @lnkuiper in #17717
• Correctly setting the delim offset by @Damon07 in #17716
• fix linux extension ci by @samansmink in #17720
• Aggregation performance by @lnkuiper in #17718
• Fix windows-2025 build errors by @adsharma in #17726
• [SQLLogicTester] Introduce reset label <query label> in the tester by @Tishj in #17729
• Adding additional authenticated data for encryption by @ccfelius in #17508
• csv_scanner: correct code comment by @Djfe in #17735
• Deprecate windows-2019 runners by @hannes in #17745
• re-add httpfs apply_patches by @samansmink in #17755
• Rename decorator from test_nulls to null_test_parameters by @Mytherin in #17760
• [CAPI] Expose ErrorData by @taniabogatsch in #17722
• Expose file_size_bytes and footer_size in parquet_file_metadata by @gijshendriksen in #17750
• Pass ExtensionLoader when loading extensions, change extension entry function by @Maxxen in #17772
• Support glibc 2.28 environments by @James-Gilbert- in #17776
• Mark Upper/LowerComparisonType as const by @JelteF in #17773
• [Indexes] Buffer-managed indexes part 1: segment handles by @taniabogatsch in #17758
• [Julia] api docs improvements by @tqml in #15645
• Ensure we use the same layout in RadixPartitionedHashTable and GroupedAggregateHashTable by @lnkuiper in #17790
• [Profiling] Propagate the ClientContext into file handle writes by @taniabogatsch in #17754
• Fix propagatesNullValues for case expr by @suibianwanwank in #17796
• Add qualified parameter to Python GetTableNames API by @evertlammerts in #17797
• Merge v1.3 into main by @Mytherin in #17806
• Pushdown pivot filter by @flashmouse in #17801
• Replace string for const data ptr in encryption api by @ccfelius in #17825
• Merge130 by @carlopi in #17833
• fix: escape using_columns on JoinRef::ToString by @akoshchiy in #17839
• Fix ICE with Windows ARM64 by @staticlibs in #17844
• Merge v1.3 into main by @Mytherin in #17851
• Add duckdb_type column to parquet_schema by @Mytherin in #17852
• Internal #4991: Remove Epoch_MS(MS) by @hawkfish in #17816
• #17853 Enable flexible page sizes and update Android NDK to r27 in workflow. by @aprock in #17854
• [Indexes] Buffer-managed indexes part 2: segment handle for base nodes by @taniabogatsch in #17828
• Function Serialization: adapt to removal of overloads by explicitly casting if argument types have changed by @Mytherin in #17864
• julia: add missing methods from C-API by @tqml in #17733
• Issue #17153: Window Order Columns by @hawkfish in #17835
• Issue #17040: FILL Secondary Sorts by @hawkfish in #17821
• Add STRUCT to MAP cast function by @evertlammerts in #17799
• Issue #17849: Test FILL Duplicates by @hawkfish in #17869
• Add GenAI policy by @szarnyasg in #17882
• Update function descriptions and examples for list, array, lambda functions by @c-herrewijn in #17886
• Issue #17861: FILL Argument Types by @hawkfish in #17888
• Reword GenAI policy by @szarnyasg in #17895
• Use an arena linked list for the physical operator children by @taniabogatsch in #17748
• Make CTE Materialization the Default Instead of Inlining by @kryonix in #17459
• Merge v1.3 into main by @Mytherin in #17897
• Leverage VectorType in ColumnDataCollection by @lnkuiper in #17881
• Fix empty BP block when writing parquet by @platypii in #17929
• fix use after free in adbc on invalid stmt by @ruslandoga in #17927
• Do not dispatch JDBC/ODBC jobs in release CI runs by @staticlibs in #17937
• Block based encryption by @ccfelius in #17275
• Unittester failures summary by @hmeriann in #16833
• Add v1.3-ossivalis to Cross version workflow by @hmeriann in #17906
• [CI Nightly Fix] Skip logging test if not standard block size by @taniabogatsch in #17957
• Visual Studio 17 (2022) fixes by @edouarda in #17948
• [Nested] Add struct_position and struct_contains functions by @maiadegraaf in #17819
• Enable building spatial and encodings extensions by @staticlibs in #17960
• [Nested] Optimize structs in LIST_VALUE by @maiadegraaf in #17169
• Unit Tester Configuration by @Mytherin in #17972
• [nested] Allow fixed-size arrays to be unnested by @maiadegraaf in #17968
• Merge v1.3-ossivalis into main by @Mytherin in #17973
• [CI] Skip some workflows when updating out of tree extensions SHA by @hmeriann in #17949
• Issue #5144: AsOf Join Threshold by @hawkfish in #17979
• [Fix] Reset profiling info before preparing a query by @taniabogatsch in #17940
• Flag to disable database invalidation by @taniabogatsch in #17938
• Issue #5123: make_timestamp_ms by @hawkfish in #17908
• Rework extension loading to go through thread-safe ExtensionManager by @Mytherin in #17994
• Implement consumption and production of Arrow Binary View by @pdet in #17975
• Add support to produce Polars Lazy Dataframes by @pdet in #17947
• c-api to copy vector with selection by @abramk in #17870
• Fix #18007: correctly execute expressions with pivot operator by @Mytherin in #18020
• [Chore] Minor conflict manager refactoring by @taniabogatsch in #18015
• Remove Linux (32 Bit) job by @hmeriann in #18012
• [Explain] Add the YAML format for EXPLAIN statements by @qsliu2017 in #17572
• Unittest: Add skip_compiled option that can be used to skip built-in C++ tests by @Mytherin in #18034
• Add ppc64le spin-wait instruction by @mgiessing in #17837
• Merge ossivalis into main by @taniabogatsch in #18036
• Remove match-case statements from polars_io.py by @evertlammerts in #18052
• Avoid adding commands read from a file to the shell history by @Mytherin in #18057
• Adding WAL encryption by @ccfelius in #17955
• Encryption: adding -key for the command line by @ccfelius in #17950
• fix star expr exclude error by @jayhan94 in #18063
• Add support for class-based expression iteration by @Mytherin in #18070
• Use timestamp_t instead of time_t for file last modified time by @lnkuiper in #18037
• Unittester: add on_new_connection + on_load + skip_tests options by @carlopi in #18042
• Fix some scaling issues by @lnkuiper in #17985
• Issue #18071: Temporal inf -inf by @hawkfish in #18083
• Switch to Optional for type hints in polars lazy dataframe function by @evertlammerts in #18078
• Unittest: Configure skip error messages by @carlopi in #18087
• Avoid running DraftPR.yml until timeout if token is missing by @carlopi in #18090
• Add start/end offset percentage options to Python test runner by @Flogex in #18091
• [CSV Reader] Prohibit options delim and sep in same read_csv call by @ackxolotl in #18096
• Fix correlated subquery unnest fail by @flashmouse in #18092
• [CI] don't run jobs on draft PRs by @hmeriann in #18016
• TPC-DS: Use BIGINT fields by @szarnyasg in #18098
• Don't throw InternalException in Sort::Sink by @lnkuiper in #18105
• ci: build duckdb against the latest emscripten by @cpcloud in #18110
• [chore] Merge v1.3-ossivalis on main by @carlopi in #18109
• Update description of 'arrow_lossless_conversion' by @szarnyasg in #18046
• Internal #3273: Window Task Generation by @hawkfish in #18113
• set ::error:: annotations for test runners by @hmeriann in #18072
• Improve sort key comparison performance by @lnkuiper in #18131
• Add support for MERGE INTO by @Mytherin in #18135
• Detect when updates have no effect, and skip performing the actual updates if we encounter these nop updates by @Mytherin in #18144
• Add support for AdbcConnectionGetObjects(table_type) by @kou in #18066
• Issue #17683: TIME_NS Compilation by @hawkfish in #18053
• Implement replace_type function by @lnkuiper in #18077
• Bump spatial again: include re-linking for handling global objects in Wasm by @carlopi in #18170
• Resolve some small build issues by @madscientist in #18162
• fix typo by @felixhummel in #18165
• Avoid realloc in CSV writer by @lnkuiper in #18174
• fix bug with allowed_paths by @samansmink in #18176
• Reduce lock contention for the instance cache by @NiclasHaderer in #18079
• Check if GetLastSegment is not nullptr in ColumnData::RevertAppend by @lnkuiper in #18171
• [Profiling] Move the client context into more write functions by @taniabogatsch in #17875
• Bump Julia to v1.3.2 by @hmeriann in #18185
• Merge v1.3-ossivalis into main by @carlopi in #18188
• Parquet reader logging by @lnkuiper in #18172
• Add VS2019 compat flag to Python wheel build by @staticlibs in #18198
• [Parquet][Dev] Update the vendored parquet.thrift to 3ce0760 by @Tishj in #18195
• Two-rowID-leaf support in the conflict manager and general refactoring by @taniabogatsch in #18194
• More internal-linkage by @Maxxen in #18177
• Temporary file encryption by @Mytherin in #18208
• Adding temporary file encryption by @ccfelius in #18013
• Skip logging test for smaller block sizes by @taniabogatsch in #18201
• ci(pyodide): enable WASM exceptions on the latest pyodide build by @cpcloud in #18173
• Allow explicit compression for user types by @lnkuiper in #18219
• Get type of encoded SortKey from TupleDataLayout by @lnkuiper in #18218
• Improve Parquet reader NULL statistics and compress all-NULL columns using CompressedMaterialization by @lnkuiper in #18217
• Internal #5264: NLJ Not Distinct by @hawkfish in #18216
• Bug#18163 Fix STDDEV_SAMP undeterminism by @minaracic in #18210
• [Parquet] Add read support for the VARIANT LogicalType by @Tishj in #18187
• Track DataChunk memory usage in various places by @lnkuiper in #18191
• Better NULL handling in TupleDataLayout by @lnkuiper in #18069
• Dictionary functions by @lnkuiper in #18127
• Add support for geoarrow encoded geometries in geoparquet files. by @cfis in #17942
• Improve descriptions of thresholds vars affecting join algorithm selection by @TheHillBright in #17377
• Connect relations that are not in a subgraph, but are still part of the new relation set by @Tmonster in #18182
• [Fix] Don't write empty (partial) blocks for FSST dictionary compression by @taniabogatsch in #18203
• Slightly higher memory limit for test by @lnkuiper in #18235
• Re-add string -> hugeint compressed materialization function by @lnkuiper in #18234
• [Fix] Database path conflict resolution by @taniabogatsch in #18247
• Remove require block size from a batch of tests by @taniabogatsch in #18242
• Add nightly builds for out-of-tree python extension by @evertlammerts in #18239
• Backport DB invalidation flag to ossivalis by @taniabogatsch in #18244
• Bump vcpkg-duckdb-ports and test extensions on Windows 10 default stdlib by @carlopi in #18205
• Add type safety to FlatVector::GetData<T>, ConstantVector::GetData<T> and UnifiedVectorFormat::GetData<T> by @Mytherin in #18256
• [Fix] Adjust test for smaller block sizes by @taniabogatsch in #18255
• Fix integer overflow in sequence vector by @xuke-hat in #18245
• fixes for some minor llvm 20 complaints by @hannes in #18257
• update run_extension_medata_tests.sh by @hmeriann in #17976
• Bunch of loosely connected test/CI fixes by @carlopi in #18254
• disable WebAssembly duckdb-wasm builds job in NightlyTests triggered by 'workflow_dispatch' event by @hmeriann in #18129
• Allow for static libs from extension dependencies to be bundled by @abramk in #18226
• Fix dictionary-related assertions by @lnkuiper in #18260
• Fixes for gcc 15 by @hannes in #18261
• Reduce copy in Vector::Reinterpret by @xuke-hat in #18264
• [Parquet] Add read support for the VARIANT LogicalType (with shredded encoding) by @Tishj in #18224
• Expanded autocomplete suggestions by @Dtenwolde in #18243
• Support HUGEINT in printf and format by @xuke-hat in #13277
• Move aarch64 / arm64 to native github runner by @evertlammerts in #18269
• Bump vcpkg-duckdb-ports to solve OSX linking by @carlopi in #18268
• Add support for RETURNING to MERGE INTO by @Mytherin in #18271
• Use set for row ID scanning during index scans by @taniabogatsch in #18274
• Use DuckDB cast infrastructure in fmt for new uhugeint/hugeint code by @Mytherin in #18275
• [Fix] Adjust test to run with different block sizes by @taniabogatsch in #18277
• Use FromEpochSeconds instead of FromTimeT in FileSystem::GetLastModifiedTime by @lnkuiper in #18281
• Add target for installing Python deps. by @xevix in #18285
• backport 'Unit Tester Configuration' pt2 by @c-herrewijn in #18282
• backport 'Unit Tester Configuration' by @c-herrewijn in #18263
• Fixup Main.yml for v1.3-ossivalis post #18282 by @carlopi in #18289
• SHOW TABLES FROM <qualified_name> by @xevix in #18179
• [Unittester] Add autoloading option by @carlopi in #18290
• Merge ossivalis into main by @taniabogatsch in #18272
• resolve hidden merge conflict with duplicate db name in json configs by @c-herrewijn in #18292
• Bump vcpkg-duckdb-ports, now fixing also mingw by @carlopi in #18300
• [Fix] Missing block when renaming fields by @taniabogatsch in #18293
• [Arrow] Fix unused static function warning by @Tishj in #18278
• feat: Parquet extension add row_group_compressed_size by @mapleFU in #18294
• [Parquet][Write] Fix timestamp sec writes to parquet by @Tishj in #18273
• bump httpfs by @Tmonster in #18258
• [Clang Tidy] Fix missing includes in patas_scan.hpp by @Tishj in #18276
• New Arrow C-API by @pdet in #18246
• Skip test/sql/copy/s3/url_encode.test due to httpfs update by @carlopi in #18317
• Make storage-version a test parameter by @Mytherin in #18324
• Backport #18254 by @carlopi in #18306
• feat: making Parquet write RowGroup.total_compressed_size by @mapleFU in #18307
• add the from-table-function as parameter to copy-from-bind by @peterboncz in #18004
• Python external dispatch param fixes by @evertlammerts in #18343
• Aarch64 backport by @evertlammerts in #18345
• Fix debug error in join order optimizer by @Tmonster in #18344
• [Fix] Block verification for add and drop field info by @taniabogatsch in #18302
• download Real Nest data in quiet mode by @hmeriann in #18346
• Fix condition indexes in join filter pushdown by @Damon07 in #18341
• [unittest] - fix doubled error headers on Unexpected failure by @hmeriann in #18314
• Extend PEG parser grammar by @Dtenwolde in #18221
• [C API] Expose expressions and use them in scalar function binding by @taniabogatsch in #18142
• Restore OSX tests, moving them to single --autoloading available step by @carlopi in #18335
• Add support for checkpointing in-memory tables by @Mytherin in #18348
• Revert "[unittest] - fix doubled error headers on Unexpected failure" by @Mytherin in #18355
• Python external dispatch param fixes by @evertlammerts in #18359
• Re-enable url-encode test by @Tmonster in #18360
• Enable stack traces on linux for bundled libraries by @taniabogatsch in #18363
• Split up out-of-tree extensions into separate files, and allow out-of-tree extensions to be built using BUILD_EXTENSIONS={ext_name} by @Mytherin in #18357
• Pass AttachOptions to attach method, and turn StorageExtensionInfo into an optional_ptr by @Mytherin in #18368
• Merge v1.3-ossivalis into main by @carlopi in #18364
• More robustness around deprecated extension settings by @carlopi in #18353
• Add missing ninja to workflow file by @Dtenwolde in #18373
• bump httpfs by @Tmonster in #18380
• Re-enable but deprecate CORE_EXTENSIONS in CMakeLists.txt by @evertlammerts in #18377
• Uncomment skipped decimal REE tests by @amoeba in #18372
• add option 'block_size' to test configs by @c-herrewijn in #18347
• [chore] Fixup side-effects from 8cf9ed4 by @carlopi in #18385
• Bump httpfs by @taniabogatsch in #18388
• Re-use table metadata when table is not altered during checkpoint by @Mytherin in #18390
• Approx database count system function by @taniabogatsch in #18392
• Re-use metadata of unaltered row groups when checkpointing a table by @Mytherin in #18395
• Correct and consistent integer arithmetic error messages by @soerenwolfers in #18393
• Record whether or not cross products are implicit or not, and use this for converting queries back to SQL by @Mytherin in #18394
• CI: Fix Discussion mirroring by @szarnyasg in #18397
• Store extra metadata blocks in RowGroupPointer, and only flush dirty Metadata blocks by @Mytherin in #18398
• Internal #3273: Window Hashed Sort by @hawkfish in #18337
• Wrap runner.ExecuteFile, otherwise cleanup is not properly performed by @carlopi in #18400
• [BUGFIX] Update delim offset for RHS of DELIM JOIN when correlated column is in RHS of Cross product by @Tmonster in #18375
• CI: Add separate job for discussion mirroring by @szarnyasg in #18407
• [ Python SQLLogic Tester ] Add MERGE_INTO statement to duckdb python by @hmeriann in #18402
• Remove incorrect assertion by @Mytherin in #18404
• Internal #5294: TIME_NS C API by @hawkfish in #18215
• Add DuckLake back in by @Mytherin in #18405
• Add support for table_constraints of AdbcConnectionGetObjects() by @kou in #18181
• Merge v1.3-ossivalis in main by @carlopi in #18401
• feat: remove anything following ? in database name by @rustyconover in #18417
• Correctly fetch only base column data in ColumnData::FetchUpdateData by @Mytherin in #18423
• Refactor extension CI to use extension-ci-tools by @samansmink in #18361
• Internal #5367: SortedAggregateFunction Sort Update by @hawkfish in #18408
• Internal #5368: WindowNaiveAggregator Sort Update by @hawkfish in #18409
• [Fix] Block size nightly by @taniabogatsch in #18425
• [Chore] Tidy test configs by @taniabogatsch in #18426
• Include pyodide build configuration by @rgbkrk in #18183
• Parquet: add row-group ordinal during writing encryption by @mapleFU in #18433
• [Fix] Reset segment memory when initialising new Prefix by @taniabogatsch in #18441
• Update pyodide build to 0.28.0 by @rgbkrk in #18446
• Add support for "template" types by @Maxxen in #18410
• Internal #5384: WindowDistinctAggregator Sort Update by @hawkfish in #18442
• [Chore] Improve skipped tests in test config and add verify_fetch_row config by @taniabogatsch in #18436
• Buffer index appends during WAL replay by @taniabogatsch in #18313
• Add support for generic settings, and move many settings over to generic settings by @Mytherin in #18447
• Internal #5385: WindowMergeSortTree Sort Update by @hawkfish in #18461
• Bump postgres to latest main by @Mytherin in #18464
• Merge ossivalis by @taniabogatsch in #18456
• Internal #5366: WindowDeltaScanner by @hawkfish in #18468
• SUM and + Operator for Varints by @pdet in #18424
• [Fix] Rework transaction logic in commit, rollback and checkpoint paths by @taniabogatsch in #18474
• re-nable extensions in invokeci by @samansmink in #18476
• Internal #5384: Window Sorting Polish by @hawkfish in #18484
• Unify ON CONFLICT and MERGE INTO by @Mytherin in #18480
• More insights around dict_fsst compression failure by @taniabogatsch in #18475
• Change ctrl-a/ctrl-e to move to start/end of line, not buffer by @tpot in #18490
• add delta linux back to ci by @samansmink in #18491
• Fix accidental internal exception in type transformation by @hannes in #18492
• [Profiling] Add client context into read functions by @maiadegraaf in #18438
• julia: config improvements by @aplavin in #17585
• fix: add missing space in AttachInfo::ToString() by @rustyconover in #18500
• Merge ossivalis by @taniabogatsch in #18502
• Change UNICODE to UTF8 by @sheldonrobinson in #17586
• Fix: Remove overly strict assertion on empty string value by @Dtenwolde in #18504
• Fix several bugs/fuzzer issues by @Mytherin in #18503
• Allow expressions to be used in ATTACH / COPY options by @Mytherin in #18515
• Remove immediate_transaction_mode from DB config options by @jeewonhh in #18516
• Temporarily excluding Build Pyodide wheel for Python 3.11 because it fails to build WASM wheels by @hmeriann in #18508
• ParserException for Pragma with named parameters by @Dtenwolde in #18506
• Add verify fetch row config to Main.yml by @taniabogatsch in #18478
• Adding WITH ORDINALITY to DuckDB by @niykko in #16581
• When tracking evicted_data_per_tag, track actual size on disk after temp file compression by @Mytherin in #18521
• Fix: Write the salt together with the HT offset when determining the value for key comparison by @gropaul in #18374
• Fix incorrect character encoding in GetLastErrorAsString on Windows by @soutong in #18431
• Dynamically determine dictionary size limit in Parquet writer (if unset) by @lnkuiper in #18356
• Internal #16560: Numeric TRUNC Precision by @hawkfish in #18511
• Consistently detect JSON schema indepent of number of threads by @lnkuiper in #18522
• ALP test: skip TPC-DS 67 - it is not consistent with floating point numbers by @Mytherin in #18528
• [Varint] Negation, Subtraction and Over/under-flow checking by @pdet in #18477
• fix: support both field orders for variant struct by @samansmink in #18532
• Add CAPI to retrieve client context for table functions by @VGSML in #18520
• Add StatementVerifier for EXPLAIN by @lnkuiper in #18529
• Use global index, not local id when creating filters in MultiFileColumnMapper by @lnkuiper in #18537
• Add support for explicit clean-up routine in test config, and exit multi-statement execution when an error is encountered by @Mytherin in #18539
• fix: improve handling variant nulls and nested types by @samansmink in #18538
• Allow overriding openssl version for FIPS compliance by @abramk in #18499
• Unittester: Add the --sort-style parameter that allows a fallback comparison where results are sorted according to a given sort-style by @Mytherin in #18542
• Restore missing test/configs/small_block_size.json file by @hmeriann in #18507
• [Fix] Follow-up PR to only delete unique row IDs by @taniabogatsch in #18545
• [ART] Node::Free refactoring by @taniabogatsch in #18544
• Implement special-case VARCHAR to JSON[] casts and vice versa by @lnkuiper in #18541
• Check if heap_block_ids is empty before getting start/end when destroying chunks in TupleDataCollection by @lnkuiper in #18556
• optimize/parquet: generate movable types for parquet by @mapleFU in #18510
• [easy] [no-op] Minor optimization on iterator lookup by @dentiny in #15349
• Fixing compilation with -std=cpp23 by @hannes in #18557
• Add compile option standalone-debug for clang by @flashmouse in #17433
• Rename the Varint type to Bignum by @pdet in #18547
• [Indexes] Buffer-managed indexes part 3: segment handle for Node48 and Node256 by @taniabogatsch in #18567
• fix: add formatting to explain row counts by @rustyconover in #18566
• [CSV Sniffer] Fixing bug of not properly setting skipped rows from sniffer by @pdet in #18555
• [Fix] Tidy check ossivalis by @taniabogatsch in #18583
• [Fix] Adjust shrink threshold back to original count > SHRINK_THRESHOLD by @taniabogatsch in #18582
• Flip left/right delim join based on cardinalities by @lnkuiper in #18552
• fix: use thousands separator and decimal for row counts induckbox output format by @rustyconover in #18564
• Force LIST/ARRAY child vectors on a Parquet single page by @lnkuiper in #18578
• String dictionary hash cache by @lnkuiper in #18580
• fix: libduckdb.so missing soversion by @strophy in #18305
• Pushdown filters on coalesced outer join keys compared for equality under the join condition by @matteobilardi in #18169
• Adds a function for updating and adding values in a struct by @teaguesterling in #15533
• fix hidden merge conflict by @hannes in #18589
• Increment storage version to enable DICT_FSST in benchmark file by @lnkuiper in #18588
• [Fix] Hidden test failure in test_struct_update.test by @taniabogatsch in #18598
• correctly setting log transaction id in ThreadContext by @xuke-hat in #18536
• Backport renaming a config name small_block_size.json to block_size_16kB in NightlyTests by @hmeriann in #18581
• Update README.md by @matthew-wright07 in #18614
• [Test] Fix test case and a benchmark by @hmeriann in #18610
• [CI] Don't zip and upload Code Coverage tests results when Code Coverage got cancelled by @hmeriann in #18607
• [Profiling] Add client context into more read functions by @maiadegraaf in #18514
• bump httpfs by @Tmonster in #18591
• Fix serialization backwards compatability for varargs functions by @Maxxen in #18596
• Issue #18631: Streaming Windowed Quantile by @hawkfish in #18636
• parquet/parquet_multi_file_info.cpp: fix move from stack by @carlopi in #18634
• Adjust filter pushdown to latest polars release by @pdet in #18624
• Re-add hugeint to __internal_compress_string by @jeewonhh in #18622
• Add Field IDS to multi file reader for positional deletes by @Tmonster in #18617
• [CSV Sniffer] Fix type detection issue with union and empty columns by @pdet in #18606
• [ART] ART::Erase refactoring by @taniabogatsch in #18595
• wrap httplib ::max() call in WIN_32 check by @Tmonster in #18590
• Add enable verification config run by @taniabogatsch in #18467
• feat: add ETA to progress bar in DuckDB CLI by @rustyconover in #18575
• Add "Hash Zero" verification CI run by @lnkuiper in #18623
• Make more configs into generic settings by @jeewonhh in #18592
• bump avro to v1.4 by @Tmonster in #18434
• bump spatial (on main) by @Maxxen in #18197
• Change arrow() to export record batch reader by @pdet in #18642
• [Fix] Prevent logger deadlock by @taniabogatsch in #18637
• Remove PRAGMA enable_verification by @taniabogatsch in #18645
• Add 1.4 release codename by @hannes in #18652
• Python test runner: Fix result check for COPY ... RETURN_STATS queries by @Flogex in #18625
• Merge ossivalis into main by @taniabogatsch in #18644
• CLI: Make ETA more of an estimate, and support large_row_rendering for footers by @Mytherin in #18656
• Remove more PRAGMA enable_verification by @taniabogatsch in #18664
• [CI] skip building encodings extension in InvokeCI by @hmeriann in #18655
• Python test runner: Fix hash comparison error output by @Flogex in #18626
• [Dev] Add script to create patch from changes in an extension repository by @Tishj in #18620
• Correctly set weights in reservoir sample when switch to slow sampling by @xuke-hat in #18563
• Internal #5366: Window Interrupt Arguments by @hawkfish in #18651
• Remove PRAGMA enable_verification in more tests by @taniabogatsch in #18670
• [ Python SQLLogic Tester ] Add MERGE_INTO to statement.type enum in result.py by @hmeriann in #18675
• Load pandas in import cache before binding by @evertlammerts in #18658
• Internal #5662: IEJoin Test Plans by @hawkfish in #18680
• Correctly allocate uncompressed string data in ZSTD for many giant strings by @Mytherin in #18678
• Grab lock and double-check that column is not loaded in MoveToCollection by @Mytherin in #18677
• fix error message related to wrong memory unit by @LiranBri in #18671
• [CI] Temporarily skip triggering R Package Windows (Extensions) job by @hmeriann in #18628
• Fix the issue where delta_for isn't used in bitpacking when for is unavailable by @xuke-hat in #18616
• Add date_trunc() simplification rules by @rcurtin in #18457
• Expected errors 2053 by @hmeriann in #14213
• Internal #5366: Window State Arguments by @hawkfish in #18676
• Add WAL test config run by @taniabogatsch in #18683
• Using a different workflow to release the python package by @evertlammerts in #18685
• Make sure parse errors are wrapped in ErrorData by @evertlammerts in #18682
• [Python SQLLogicTest] Add test/sql/pragma/profiling/test_profiling_all.test to the SKIPPED_TESTS set by @hmeriann in #18689
• Issue #18457: DateTrunc Simplification Warnings by @hawkfish in #18687
• Expected errors 2053 by @hmeriann in #18695
• Hold row group lock for entire call of MoveToCollection by @Mytherin in #18694
• Unplug python (in ossivalis) by @evertlammerts in #18699
• Correctly handle collations for IN (subquery) by @Mytherin in #18698
• Move attached databases from a CatalogSet to a dedicated map of shared pointers by @Mytherin in #18693
• Make ART construction iterative via ARTBuilder by @taniabogatsch in #18702
• [Fix] Correctly handle table and index chunks in WAL replay buffering by @taniabogatsch in #18700
• Python-style positional/named arguments for macro's by @lnkuiper in #18684
• Internal #3273: Hashed Sort States by @hawkfish in #18690
• Add Option to Allocate Using an Arena in string_t by @maiadegraaf in #17992
• Fix issue with materialized CTE optimization in flatten_dependent_join by @kryonix in #18714
• [Profiling] Add Profiling to Read Function by @maiadegraaf in #18661
• Correctly throw an error when too few columns are supplied in MERGE INTO INSERT by @Mytherin in #18715
• Improved grammar generation script by @NiclasHaderer in #18716
• #Fix 18558: add row_group scan fast path by @flashmouse in #18686
• Added support for blob<->uuid conversions by @dioptre in #18027
• Minor fixes for other catalogs - mostly checking IsDuckTable() for unsupported operations by @Mytherin in #18720
• Fix PIVOT in multiple statements by @evertlammerts in #18729
• Internal #5669: Loop Join Thresholds by @hawkfish in #18733
• feat: enhance .tables command with schema disambiguation and filtering by @shivampr in #18641
• Add (CSV) file logger by @samansmink in #17692
• Use 1-based indexing for SQL-based JSON array extraction by @lnkuiper in #18735
• [unittest] SkipLoggingSameError() to make unittester report one failure per case by @hmeriann in #18270
• fix timetravel for default tables by @samansmink in #18240
• [C API] Function to set a copy callback for bind data by @taniabogatsch in #18739
• Secrets: if serialization_type is not specified, assume it's a key value secret by @Mytherin in #18743
• Merge ossivalis into main by @taniabogatsch in #18719
• Use correct type for pushing collations in subqueries by @Mytherin in #18744
• Add OS X notarization for DuckDB CLI and libduckdb.dylib by @hannes in #18747
• Add missing expected errors to the test cases by @hmeriann in #18746
• removed placeholder client directories for node and jdbc, its been > 1 yr by @hannes in #18757
• Append using a SQL query, instead of directly appending to a base table, and support user-provided queries through the QueryAppender by @Mytherin in #18738
• Backport #18374 to v1.3-ossivalis by @lnkuiper in #18752
• Add leak suppressions to nightly runs by @Mytherin in #18748
• Remove separate WAL encryption flag by @Mytherin in #18750
• Fixing lazy polars execution on query result by @pdet in #18749
• [Profiling] Add Profiling to Write Function by @maiadegraaf in #18724
• Extensions.yml should also check converted_to_draft by @carlopi in #18754
• Minor logging fixes and more benchmarking by @samansmink in #18755
• Add missing expected errors to the test cases (next chunk) by @hmeriann in #18753
• Refactor read_blob and read_text to use MultiFileFunction. by @xevix in #18706
• Add support for auto-globbing within a directory: if no matches are found for a specific path, we retry with /**/*.[ext] appended by @Mytherin in #18760
• Fix radix partitioning with more than 10 bits by @ctsk in #18761
• Fix index resolution when querying table with index via view by @mach-kernel in #18319
• Fix Path Typo in Extension's CMake Warning Message by @beryllw in #18766
• Make duckdb_log return a TIMESTAMP_TZ by @Mytherin in #18768
• Revert "Use 1-based indexing for SQL-based JSON array extraction" by @Mytherin in #18758
• [CI] Adjust test configs post logger PR by @taniabogatsch in #18771
• [Test Fix] Forward output to file by @taniabogatsch in #18772
• Propagate DUCKDB_*_VERSION in extensions and tests by @Y-- in #18774
• Add file_size_bytes (de-)serialization by @lnkuiper in #18775
• Use microsecond resolution for printing the current timestamp by @Mytherin in #18776
• Improve error messages for merge / vector reference by @Mytherin in #18777
• Move row id logic to separate RowIdColumnData class instead of inlining it into the RowGroup by @Mytherin in #18780
• Treat ENABLE_EXTENSION_AUTOINSTALL as the BOOL that it is by @evertlammerts in #18778
• Add memory_limit parameter to benchmark_runner/test_runner.py by @lnkuiper in #18790
• fix: improve speed of GetValue() for STRUCT type by @rustyconover in #18785
• Internal #3273: Parallel Window Masks by @hawkfish in #18731
• Task Scheduler: track exact task count, and re-signal on dequeue failure if there are tasks left by @Mytherin in #18792
• fix: coalesce query progress updates to reduce terminal writes by @rustyconover in #18672
• Support expressions as COPY file target by @Mytherin in #18795
• Remove everything python-package related by @evertlammerts in #18789
• Improve autocomplete suggestions by @Dtenwolde in #18773
• bump httpfs so it includes curl option by @Tmonster in #18691
• Issue #18767: Ignore Timestamp Offsets by @hawkfish in #18794
• Fixup progress_bar: avoid converting doubles into int32_t unchecked by @carlopi in #18800
• [chore] Fixup tidy-check on src/logging/log_manager.cpp by passing const & by @carlopi in #18801
• Internal #3273: Hashed Sort Callbacks by @hawkfish in #18796
• Typed macro parameters by @lnkuiper in #18786
• Fix some unindented interactions between EMPTY_RESULT_PULLUP and MATERIALIZED CTEs by @kryonix in #18805
• Add support for non-aggregate window functions by @Maxxen in #18788
• Expected errors 2053 by @hmeriann in #18810
• Test runner: Expand '{UUID}' into a random UUID by @carlopi in #18809
• Provide failing file name in Parquet reader error messages by @Mytherin in #18814
• [CI] install libcurl4-openssl-dev with apt-get by @hmeriann in #18811
• fix: Add COLLATE NOCASE support to strpos function by @shivampr in #18819
• Add callback to get a list of copy options, use this to provide suggestions and to erase options from import that are only used during exporting by @Mytherin in #18812
• For BC reasons - keep VARINT as alias for BIGNUM by @Mytherin in #18821
• [Fix] Bug in fixed-size buffer when throwing out-of-memory by @taniabogatsch in https://github.com/duckdb/duckdb/pull/18769
• Re-add accidentally removed check if copy_from is supported by @Mytherin in https://github.com/duckdb/duckdb/pull/18824
• Fix format-fix runs on Linux by @staticlibs in https://github.com/duckdb/duckdb/pull/18827
• Extensions.yml: Pass down save_cache to inner workflows by @carlopi in https://github.com/duckdb/duckdb/pull/18828
• Fix: Preserve database configuration flags for tab completion in DuckDB shell by @rustyconover in https://github.com/duckdb/duckdb/pull/18482
• Ensure a WAL file matches the DB file and checkpoint iteration by @taniabogatsch in https://github.com/duckdb/duckdb/pull/18823
• fix: sanitize input for enable_logging by @samansmink in https://github.com/duckdb/duckdb/pull/18830
• fix: silence warnings about signed/unsigned conversions. by @rustyconover in https://github.com/duckdb/duckdb/pull/18835
• Avoid expensive checkpoints and write amplification by appending row groups, and limiting vacuum operations for the last number of row groups by @Mytherin in https://github.com/duckdb/duckdb/pull/18829
• Fix/run function in transaction by @Evannnnnnnn in https://github.com/duckdb/duckdb/pull/18741
• add appender to concurrent test by @c-herrewijn in https://github.com/duckdb/duckdb/pull/18721
• Add support for reading/writing native parquet geometry types by @Maxxen in https://github.com/duckdb/duckdb/pull/18832
• Don't notify Py pkg when override git describe is set by @evertlammerts in https://github.com/duckdb/duckdb/pull/18843
• Avoid printing '99 hours', given in most cases that means estimate is… by @carlopi in https://github.com/duckdb/duckdb/pull/18839
• Add the VARIANT LogicalType by @Tishj in https://github.com/duckdb/duckdb/pull/18609
• Document storage version flag in CLI + minor rendering fix by @Mytherin in https://github.com/duckdb/duckdb/pull/18841
• Ignore null verification for statistics on structs by @d-justen in https://github.com/duckdb/duckdb/pull/18813
• Add OnBeginExtensionLoad callback by @Mytherin in https://github.com/duckdb/duckdb/pull/18842
• Bump MySQL/Postgres/SQLite by @Mytherin in https://github.com/duckdb/duckdb/pull/18848
• Merge ossivalis into main by @taniabogatsch in https://github.com/duckdb/duckdb/pull/18844
• Add test_env to unit tester by @pdet in https://github.com/duckdb/duckdb/pull/18847
• WAL <> DB File Match Fixes by @Mytherin in https://github.com/duckdb/duckdb/pull/18849
• Make ATTACH OR REPLACE atomic, keep list of used databases in MetaTransaction by @Mytherin in https://github.com/duckdb/duckdb/pull/18850
• Fix NULL path for json_each/json_tree by @lnkuiper in https://github.com/duckdb/duckdb/pull/18852
• No more wal_encryption flag by @jeewonhh in https://github.com/duckdb/duckdb/pull/18851
• Bump Ducklake by @pdet in https://github.com/duckdb/duckdb/pull/18825
• Add more encryption modes CTR and CBC by @hannes in https://github.com/duckdb/duckdb/pull/18619
• Centralize attached database paths in a DatabaseFilePathManager which is shared across databases created through the same DBInstanceCache by @Mytherin in https://github.com/duckdb/duckdb/pull/18857
• Hold segment lock during GetColumnSegmentInfo by @Mytherin in https://github.com/duckdb/duckdb/pull/18859
• update duckdb azure extension ref for 1.4.0 by @benfleis in https://github.com/duckdb/duckdb/pull/18868
• Expected errors 2053 by @hmeriann in https://github.com/duckdb/duckdb/pull/18864
• Add a FORCE_DEBUG flag to force -DDEBUG, similar to FORCE_ASSERT by @Mytherin in https://github.com/duckdb/duckdb/pull/18872
• Bump & remove patches for delta, avro, excel, encodings, fts by @samansmink in https://github.com/duckdb/duckdb/pull/18869
• [minor] Incompatible DB error message: add newline by @carlopi in https://github.com/duckdb/duckdb/pull/18861
• Bump mbedtls to v3.6.4 by @Mytherin in https://github.com/duckdb/duckdb/pull/18871
• Storage fuzzing + several fixes by @Mytherin in https://github.com/duckdb/duckdb/pull/18876
• Update ducdkb iceberg hash by @Tmonster in https://github.com/duckdb/duckdb/pull/18873
• [Test] Small fixes to concurrent attach/detach test by @taniabogatsch in https://github.com/duckdb/duckdb/pull/18862
• Internal #5796: Window Progress by @hawkfish in https://github.com/duckdb/duckdb/pull/18860
• Add COPY (...) TO ... (FORMAT BLOB) by @Maxxen in https://github.com/duckdb/duckdb/pull/18840
• Update spatial+vss+sqlsmith in preparation for v1.4 by @Maxxen in https://github.com/duckdb/duckdb/pull/18882
• Avoid automatically checkpointing if the database instance has been invalidated by @Mytherin in https://github.com/duckdb/duckdb/pull/18881
• Add COPY (FORMAT BLOB) to Andium too :^) by @Maxxen in https://github.com/duckdb/duckdb/pull/18884
• [C API] Result schema of prepared statements by @hrl20 in https://github.com/duckdb/duckdb/pull/18779
• Json: no reinterpret<size_t*> by @carlopi in https://github.com/duckdb/duckdb/pull/18886
• [Dev] Fix footgun in string_t::SetSizeAndFinalize by @Tishj in https://github.com/duckdb/duckdb/pull/18885
• [chore] Bump config test/configs/compressed_in_memory.json to new format by @carlopi in https://github.com/duckdb/duckdb/pull/18888
• bump aws and iceberg by @Tmonster in https://github.com/duckdb/duckdb/pull/18889
• Add rowsort to upsert_default.test by @jeewonhh in https://github.com/duckdb/duckdb/pull/18890
• fixing auto-specifying ciphers and remove double storage by @hannes in https://github.com/duckdb/duckdb/pull/18891
• Expected errors 2053 by @hmeriann in https://github.com/duckdb/duckdb/pull/18892
• Keep base data scan state alive in ColumnData::Update call by @Mytherin in https://github.com/duckdb/duckdb/pull/18893
• Add callback for when an extension fails to load, and also log this by @Mytherin in https://github.com/duckdb/duckdb/pull/18894
• Encryption now encoded as a bit, centralizing in set/getter by @carlopi in https://github.com/duckdb/duckdb/pull/18897
• Bump httpfs to v1.4-andium branch by @carlopi in https://github.com/duckdb/duckdb/pull/18898
• fix: refine query ETA display and Kalman filter stability by @rustyconover in https://github.com/duckdb/duckdb/pull/18880
• Bump inet & aws by @samansmink in https://github.com/duckdb/duckdb/pull/18899
• [chore] Fix amalgamation build in progress_bar by @carlopi in https://github.com/duckdb/duckdb/pull/18910
• Cannot create table from variant yet by @Mytherin in https://github.com/duckdb/duckdb/pull/18912
• In VerifyZeroReaders, get the header size from the buffer we are replacing instead of from the block manager by @Mytherin in https://github.com/duckdb/duckdb/pull/18909
• Fix #18152: avoid auto-detecting hive partitioning with COPY .. FROM by @Mytherin in https://github.com/duckdb/duckdb/pull/18911
• CLI: Correctly move to start of line by @Mytherin in https://github.com/duckdb/duckdb/pull/18920
• Strip question mark parameters from default temporary directory by @Mytherin in https://github.com/duckdb/duckdb/pull/18915
• Move Hash Zero CI run to nightly by @Mytherin in https://github.com/duckdb/duckdb/pull/18925
• Bump Iceberg by @Tmonster in https://github.com/duckdb/duckdb/pull/18917
• Issue template: Add the Python repository by @szarnyasg in https://github.com/duckdb/duckdb/pull/18928
• fix extension size increase by @samansmink in https://github.com/duckdb/duckdb/pull/18923
• [Dev] Fix reference of uninitialized memory in Variant conversion first pass by @Tishj in https://github.com/duckdb/duckdb/pull/18921
• Bump DuckLake to Latest Main by @pdet in https://github.com/duckdb/duckdb/pull/18926
• Make row-group metadata re-use experimental for now by @Mytherin in https://github.com/duckdb/duckdb/pull/18922
• Fix exception propagation in C API by @mlafeldt in https://github.com/duckdb/duckdb/pull/18924
• Bump httpfs by @carlopi in https://github.com/duckdb/duckdb/pull/18930
• Bump ducklake and don't write empty bbox in geoparquet stats by @Maxxen in https://github.com/duckdb/duckdb/pull/18936
• [PROFILING] Fix EXPLAIN ANALYZE returning empty results when PRAGMA enabled_profiling = 'no_output' by @maiadegraaf in https://github.com/duckdb/duckdb/pull/18935
• Http_util can return success for all [200, 300) responses, as well as redirects by @Tmonster in https://github.com/duckdb/duckdb/pull/18940
• Fix TransformStringToLogicalType for enums arrays by @tdoehmen in https://github.com/duckdb/duckdb/pull/18941
• [unittester] Allow overriding data/ folder to custom location by @carlopi in https://github.com/duckdb/duckdb/pull/18929
• Unpin fixed-size sorting keys by @lnkuiper in https://github.com/duckdb/duckdb/pull/18945
• Add missing parameters to COPY ... (FORMAT JSON) by @lnkuiper in https://github.com/duckdb/duckdb/pull/18946
• Fixes for encrypted database, make cross-engine encryption work, and expand testing by @hannes in https://github.com/duckdb/duckdb/pull/18951
• fix windows linking issue ducklake by @samansmink in https://github.com/duckdb/duckdb/pull/18953
• bump iceberg by @Tmonster in https://github.com/duckdb/duckdb/pull/18957
• [SQLLogicTest] Detect errors thrown in LoadExtension of the require statement by @Tishj in https://github.com/duckdb/duckdb/pull/18950
• Don't use VectorOperations::Copy for string dictionary hashes by @lnkuiper in https://github.com/duckdb/duckdb/pull/18949
• Fix error reporting in SSLClient by @staticlibs in https://github.com/duckdb/duckdb/pull/18958
• bump spatial by @Maxxen in https://github.com/duckdb/duckdb/pull/18961
• Allow extensions to customize ATTACH OR REPLACE conflict behavior by @ywelsch in https://github.com/duckdb/duckdb/pull/18962
• Unify test runner keyword replacement, and don't run LOAD [ext] by default by @Mytherin in https://github.com/duckdb/duckdb/pull/18963
• [chore] Bump httpfs and remove patches by @carlopi in https://github.com/duckdb/duckdb/pull/18965
• Correctly update row group data pointers and root table pointer after checkpoint by @Mytherin in https://github.com/duckdb/duckdb/pull/18966
• Attach: Cleanup duplicate data path handling, and make IF NOT EXISTS no longer abort if we are adding a path with the same name by @Mytherin in https://github.com/duckdb/duckdb/pull/18974
• Bump DuckLake and HTTPFS by @pdet in https://github.com/duckdb/duckdb/pull/18975
• Issue #18971: Empty Unsorted Windows by @hawkfish in https://github.com/duckdb/duckdb/pull/18976
• Check context.interrupted flag in table scan by @Mytherin in https://github.com/duckdb/duckdb/pull/18981
• Only return cgroup memory limit if it's a sane value by @szarnyasg in https://github.com/duckdb/duckdb/pull/18668
• Macro fixes by @lnkuiper in https://github.com/duckdb/duckdb/pull/18992
• ATTACH IF NOT EXISTS - wait until database is fully attached before returning by @Mytherin in https://github.com/duckdb/duckdb/pull/18993
• WALReplay Fix: In UpdateColumn, no longer assume all updates are part of the same vector, but instead verify this and batch updates per vector by @Mytherin in https://github.com/duckdb/duckdb/pull/18999
• Bump iceberg by @Tmonster in https://github.com/duckdb/duckdb/pull/19001

Full Changelog: v1.3.2...v1.4.0