Update docs and version pins for DataFusion 52 support

Note that this does NOT introduce support for a new DataFusion version,
just reconciles the docs and pins to match the DataFusion pin

Author: Volodymyr Kot

mkdocs/docs/api.md

@@ -2123,7 +2123,7 @@ PyIceberg integrates with [Apache DataFusion](https://datafusion.apache.org/) th
 
     The integration has a few caveats:
 
-    - Only works with `datafusion == 51`, aligns with the version used in `pyiceberg-core`
+    - Only works with `datafusion == 52`, aligns with the version used in `pyiceberg-core`
     - Depends directly on `iceberg-rust` instead of PyIceberg's implementation
     - Has limited features compared to the full PyIceberg API
 

pyproject.toml

@@ -93,7 +93,10 @@ sql-sqlite = ["sqlalchemy>=2.0.18,<3"]
 gcsfs = ["gcsfs>=2023.1.0"]
 rest-sigv4 = ["boto3>=1.24.59"]
 hf = ["huggingface-hub>=0.24.0"]
-pyiceberg-core = ["pyiceberg-core>=0.5.1,<0.10.0"]
+# `pyiceberg-core` bundles a `datafusion-ffi` build, and the DataFusion FFI ABI is not stable
+# across major versions (https://github.com/apache/datafusion/issues/17374). The two ranges below
+# must name the same DataFusion major version: pyiceberg-core 0.9.x bundles datafusion-ffi 52.x.
+pyiceberg-core = ["pyiceberg-core>=0.9.0,<0.10.0"]
 datafusion = ["datafusion>=52,<53"]
 gcp-auth = ["google-auth>=2.4.0"]
 entra-auth = ["azure-identity>=1.25.1"]