Mike Nitsenko d983046f90 Extend Cube Semantic Loader functionality (#8186) ... **PR Description:** This pull request introduces several enhancements and new features to the `CubeSemanticLoader`. The changes include the following: 1. Added imports for the `json` and `time` modules. 2. Added new constructor parameters: `load_dimension_values`, `dimension_values_limit`, `dimension_values_max_retries`, and `dimension_values_retry_delay`. 3. Updated the class documentation with descriptions for the new constructor parameters. 4. Added a new private method `_get_dimension_values()` to retrieve dimension values from Cube's REST API. 5. Modified the `load()` method to load dimension values for string dimensions if `load_dimension_values` is set to `True`. 6. Updated the API endpoint in the `load()` method from the base URL to the metadata endpoint. 7. Refactored the code to retrieve metadata from the response JSON. 8. Added the `column_member_type` field to the metadata dictionary to indicate if a column is a measure or a dimension. 9. Added the `column_values` field to the metadata dictionary to store the dimension values retrieved from Cube's API. 10. Modified the `page_content` construction to include the column title and description instead of the table name, column name, data type, title, and description. These changes improve the functionality and flexibility of the `CubeSemanticLoader` class by allowing the loading of dimension values and providing more detailed metadata for each document. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>		2023-07-24 12:11:58 -07:00
..
example_data	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
acreom.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
airbyte_json.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
airtable.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
alibaba_cloud_maxcompute.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
apify_dataset.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
arxiv.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
async_html.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
aws_s3_directory.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
aws_s3_file.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
azlyrics.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
azure_blob_storage_container.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
azure_blob_storage_file.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
bibtex.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
bilibili.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
blackboard.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
blockchain.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
brave_search.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
browserless.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
chatgpt_loader.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
college_confidential.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
confluence.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
conll-u.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
copypaste.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
csv.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
cube_semantic.ipynb	Extend Cube Semantic Loader functionality (#8186)	2023-07-24 12:11:58 -07:00
datadog_logs.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
diffbot.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
discord.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
docugami.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
duckdb.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
email.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
embaas.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
epub.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
evernote.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
excel.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
facebook_chat.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
fauna.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
figma.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
geopandas.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
git.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
gitbook.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
github.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
google_bigquery.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
google_cloud_storage_directory.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
google_cloud_storage_file.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
google_drive.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
grobid.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
gutenberg.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
hacker_news.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
hugging_face_dataset.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
ifixit.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
image_captions.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
image.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
imsdb.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
index.mdx	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
iugu.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
joplin.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
jupyter_notebook.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
larksuite.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
mastodon.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
mediawikidump.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
merge_doc_loader.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
mhtml.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
microsoft_onedrive.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
microsoft_powerpoint.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
microsoft_word.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
modern_treasury.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
notion.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
notiondb.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
obsidian.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
odt.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
open_city_data.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
org_mode.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
pandas_dataframe.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
psychic.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
pyspark_dataframe.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
readthedocs_documentation.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
recursive_url_loader.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
reddit.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
roam.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
rockset.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
rst.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
sitemap.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
slack.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
snowflake.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
source_code.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
spreedly.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
stripe.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
subtitle.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
telegram.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
tencent_cos_directory.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
tencent_cos_file.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
tomarkdown.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
toml.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
trello.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
tsv.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
twitter.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
unstructured_file.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
url.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
weather.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
web_base.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
whatsapp_chat.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
wikipedia.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
xml.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
xorbits.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
youtube_audio.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00
youtube_transcript.ipynb	mv module integrations docs (#8101)	2023-07-23 23:23:16 -07:00