DATA-2937 Improve get_metadata_from_sample_data with psycopg type map… - #79
Open
Shrikey wants to merge 1 commit into
Open
DATA-2937 Improve get_metadata_from_sample_data with psycopg type map…#79Shrikey wants to merge 1 commit into
Shrikey wants to merge 1 commit into
Conversation
…ping Add a module-level TYPE_MAPPING (based on FreshServiceWrapper.TYPE_MAPPING in the warehouse repository) and use it in infer_postgres_type, so datetime, date, time, Decimal, UUID, timedelta and list values map to proper PostgreSQL types instead of falling back to text. Mapping keys are corrected to Python 3 class names (Decimal instead of decimal; long/unicode/None dropped) so lookups on type(value).__name__ actually match. float maps to numeric (not real as in the FreshService mapping) to avoid float4 precision loss. Behavior change: int now always maps to bigint (previously int for small sample values), following the mapping. Adds a unit test suite for the helper functions in wherescape/tests/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BFJonk
reviewed
Aug 12, 2026
| class TestGetMetadataFromSampleData: | ||
| def test_docstring_example(self): | ||
| sample_data = [ | ||
| {"id": 1, "name": "John", "score": 95.5, "created_at": "2025-01-01", "active": True}, |
Contributor
There was a problem hiding this comment.
thought: I would create seperate tests for type based and name based type guesses, mainly because the latter should be extended over time.
| assert {type(value).__name__ for value in samples} == set(TYPE_MAPPING) | ||
|
|
||
|
|
||
| class TestGetMetadataFromSampleData: |
Contributor
There was a problem hiding this comment.
suggestion: I would add a docstring on this class for readability / reviewablity
| assert {type(value).__name__ for value in samples} == set(TYPE_MAPPING) | ||
|
|
||
|
|
||
| class TestGetMetadataFromSampleData: |
Contributor
There was a problem hiding this comment.
issue: I'm missing a rainy day scenario, what happens if there is no expected value for a certain type? or can thus never happen?
| "datetime": "timestamp", | ||
| "timedelta": "interval", | ||
| "list": "text", | ||
| # "list": "text[]", |
Contributor
There was a problem hiding this comment.
issue: isn't the commented out version better?
Contributor
There was a problem hiding this comment.
If not, a commented on why is warranted
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…ping
Add a module-level TYPE_MAPPING (based on FreshServiceWrapper.TYPE_MAPPING in the warehouse repository) and use it in infer_postgres_type, so datetime, date, time, Decimal, UUID, timedelta and list values map to proper PostgreSQL types instead of falling back to text. Mapping keys are corrected to Python 3 class names (Decimal instead of decimal; long/unicode/None dropped) so lookups on type(value).name actually match. float maps to numeric (not real as in the FreshService mapping) to avoid float4 precision loss.
Behavior change: int now always maps to bigint (previously int for small sample values), following the mapping.
Adds a unit test suite for the helper functions in wherescape/tests/.
Issue number
{if exists provide related issue}
Expected behaviour
{what should have happened}
Actual behaviour
{what happens}
Description of fix
{small description of what fixes the issue}
Other info
{anything else that might be related/useful}