🔧 chore: Improve Document Loader and Vector Store Logging - #203
Merged
Merged
Conversation
… avoid lengthy embedding logs - Implemented a static method `_sanitize_parameters_for_logging` to truncate large values and embeddings for improved logging clarity. - Updated the `setup_query_logging` method to utilize the new sanitization method, ensuring sensitive or large data is not logged directly.
- Updated the `cleanup_temp_encoding_file` function to check that `_temp_filepath` is not None before attempting to remove the file, preventing potential errors when the attribute is present but not initialized.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
I improved temporary file cleanup reliability in the document loader and enhanced logging clarity for vector store operations.
cleanup_temp_encoding_fileto verify_temp_filepathis not None before attempting file removal_sanitize_parameters_for_loggingstatic method to ExtendedPgVector for truncating embeddings and large values in logsChange Type
Testing
Test the changes by:
Test Configuration:
DEBUG_PGVECTOR_QUERIES=truefor vector store logging testsChecklist