Pandas - dataframe useful operations
The following Youtube videos show useful operations with pandas dataframe.
Blog Posts
Recent posts
PySpark - upsert two dataframe
The following PySpark code will upsert target dataframe with source dataframe even though source column is null when key column matched.
Azure - BlobClient upload/download
The following blog show how to use Azure BlobClient to download/upload blobs
PySpark - Development Environment with Dev Container
It’s hard to setup the Spark cluster to run PySpark job locally or CI system. The following github repository will show how we can use dev container feature ...
Java - Structured Concurrency in JDK 21
As a part of loom project, JDK21 will support the structured concurrency to handle multiple concurrent tasks in a structured way.