Code Utilities

📄️ BigQuery & Spark: Efficient Data Handling with Partitioning & Incremental Loading

Introduction {#h\_01J2BE2D71XV856TPA7N792KGD}

📄️ Connection Utils

This article explores the functionalities provided by the synutils.connection module in the Syntasa application. Connection utils provide a way to interact with connection parameters within the Syntasa application. It allows you to retrieve specific values associated with a connection using Python code. 1. getConnectionParam() {#h\_01HZRX5NS38XZ0XHWY02F9F40N}

📄️ Event Store Utils

This article explores the functionalities provided by the synutils.eventstore module in the Syntasa application. These utilities offer functionalities to write data and files to an Event Store using Python.

📄️ FileSystem Utils

Syntasa provides a set of utilities for interacting with the filesystem through Python and Scala code. These utilities offer functionalities like listing files, reading file contents, copying, moving, uploading, creating directories, renaming files, and checking existence.

📄️ Notification Utils

Syntasa provides a notification utility function called sendNotification to send email notifications. This function is part of the synutils.notification module.

📄️ Secret Utils

Syntasa provides a utility class, synutils.secret, for securely retrieving sensitive credentials like Google Project ID, Google Cloud credentials, AWS Access Key, and AWS Secret Key. These credentials are essential for various tasks within the Syntasa application, such as interacting with Google Cloud Platform (GCP) services or Amazon Web Services (AWS).

📄️ Spark Utils

Spark applications often involve working with data in DataFrames. Spark Utils provides helper functions to simplify these tasks, specifically focusing on DataFrame creation and table manipulation. This article explores two such functions: createDataFrame and createTable.

📄️ UDP Utils

The Syntasa application provides UDP utilities that allow you to interact with UDP mappings. These utilities are implemented in Python & Scala and offer functions to get various details about UDP mappings.