RenΓ©'s URL Explorer Experiment


Title: GitHub - astor194/dataengineering: A Data Engineering & Machine Learning Knowledge Hub Β· GitHub

Open Graph Title: GitHub - astor194/dataengineering: A Data Engineering & Machine Learning Knowledge Hub

X Title: GitHub - astor194/dataengineering: A Data Engineering & Machine Learning Knowledge Hub

Description: A Data Engineering & Machine Learning Knowledge Hub - astor194/dataengineering

Open Graph Description: A Data Engineering & Machine Learning Knowledge Hub - astor194/dataengineering

X Description: A Data Engineering & Machine Learning Knowledge Hub - astor194/dataengineering

Opengraph URL: https://github.com/astor194/dataengineering

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository
route-controllerfiles
route-actiondisambiguate
fetch-noncev2:b26adbbb-3c09-e657-b6b1-959325932e8a
current-catalog-service-hashf3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb
request-idE9E4:6F3CF:41E4E5:59102E:6A4EB6EC
html-safe-nonce83f557091aa50f0da17ecf2e72b3b63eff165d6297b683b67ad5b057ac8f96d8
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFOUU0OjZGM0NGOjQxRTRFNTo1OTEwMkU6NkE0RUI2RUMiLCJ2aXNpdG9yX2lkIjoiNDAzMzgyMTIyMTYyMzYwOTA2OCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacbff3d52232ca7160760c0480f86b78c5810d3b863cc612af4a5d58664b277016
hovercard-subject-tagrepository:415520170
github-keyboard-shortcutsrepository,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location//
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/astor194/dataengineering
twitter:imagehttps://opengraph.githubassets.com/99ba016c383793e699009fdc21eb8e21dda549d1280f1df3f2438548666b4992/astor194/dataengineering
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/99ba016c383793e699009fdc21eb8e21dda549d1280f1df3f2438548666b4992/astor194/dataengineering
og:image:altA Data Engineering & Machine Learning Knowledge Hub - astor194/dataengineering
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None41b6ab3ba6d20a71766ac245b5a4a94c6fc672a9cd4da7d44c1b33ab8bf6a21c
turbo-cache-controlno-cache
go-importgithub.com/astor194/dataengineering git https://github.com/astor194/dataengineering.git
octolytics-dimension-user_id6895821
octolytics-dimension-user_loginastor194
octolytics-dimension-repository_id415520170
octolytics-dimension-repository_nwoastor194/dataengineering
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forktrue
octolytics-dimension-repository_parent_id296615591
octolytics-dimension-repository_parent_nwoabhishek-ch/around-dataengineering
octolytics-dimension-repository_network_root_id296615591
octolytics-dimension-repository_network_root_nwoabhishek-ch/around-dataengineering
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasee6a744804e8e70f97b4d5a18a94dcc63db22f97a
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/astor194/dataengineering#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fastor194%2Fdataengineering
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub Copilot appDirect agents from issue to mergehttps://github.com/features/ai/github-app
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
View all resourceshttps://github.com/resources
GitHub SponsorsFund open source developershttps://github.com/open-source/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/open-source/accelerator
GitHub Starshttps://stars.github.com
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fastor194%2Fdataengineering
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E&source=header-repo&source_repo=astor194%2Fdataengineering
Reloadhttps://github.com/astor194/dataengineering
Reloadhttps://github.com/astor194/dataengineering
Reloadhttps://github.com/astor194/dataengineering
astor194 https://github.com/astor194
dataengineeringhttps://github.com/astor194/dataengineering
abhishek-ch/around-dataengineeringhttps://github.com/abhishek-ch/around-dataengineering
Notifications https://github.com/login?return_to=%2Fastor194%2Fdataengineering
Fork 0 https://github.com/login?return_to=%2Fastor194%2Fdataengineering
Star 1 https://github.com/login?return_to=%2Fastor194%2Fdataengineering
Code https://github.com/astor194/dataengineering
Pull requests 0 https://github.com/astor194/dataengineering/pulls
Actions https://github.com/astor194/dataengineering/actions
Projects https://github.com/astor194/dataengineering/projects
Security and quality 0 https://github.com/astor194/dataengineering/security
Insights https://github.com/astor194/dataengineering/pulse
Code https://github.com/astor194/dataengineering
Pull requests https://github.com/astor194/dataengineering/pulls
Actions https://github.com/astor194/dataengineering/actions
Projects https://github.com/astor194/dataengineering/projects
Security and quality https://github.com/astor194/dataengineering/security
Insights https://github.com/astor194/dataengineering/pulse
https://github.com/astor194/dataengineering
Brancheshttps://github.com/astor194/dataengineering/branches
Tagshttps://github.com/astor194/dataengineering/tags
https://github.com/astor194/dataengineering/branches
https://github.com/astor194/dataengineering/tags
216 Commitshttps://github.com/astor194/dataengineering/commits/master/
https://github.com/astor194/dataengineering/commits/master/
.github/ISSUE_TEMPLATEhttps://github.com/astor194/dataengineering/tree/master/.github/ISSUE_TEMPLATE
.github/ISSUE_TEMPLATEhttps://github.com/astor194/dataengineering/tree/master/.github/ISSUE_TEMPLATE
docshttps://github.com/astor194/dataengineering/tree/master/docs
docshttps://github.com/astor194/dataengineering/tree/master/docs
paperhttps://github.com/astor194/dataengineering/tree/master/paper
paperhttps://github.com/astor194/dataengineering/tree/master/paper
sketchnoteshttps://github.com/astor194/dataengineering/tree/master/sketchnotes
sketchnoteshttps://github.com/astor194/dataengineering/tree/master/sketchnotes
.gitignorehttps://github.com/astor194/dataengineering/blob/master/.gitignore
.gitignorehttps://github.com/astor194/dataengineering/blob/master/.gitignore
A Data Engineering Story.pptxhttps://github.com/astor194/dataengineering/blob/master/A%20Data%20Engineering%20Story.pptx
A Data Engineering Story.pptxhttps://github.com/astor194/dataengineering/blob/master/A%20Data%20Engineering%20Story.pptx
README.mdhttps://github.com/astor194/dataengineering/blob/master/README.md
README.mdhttps://github.com/astor194/dataengineering/blob/master/README.md
WhyDataOrchestration.pdfhttps://github.com/astor194/dataengineering/blob/master/WhyDataOrchestration.pdf
WhyDataOrchestration.pdfhttps://github.com/astor194/dataengineering/blob/master/WhyDataOrchestration.pdf
flink_cdc.mdhttps://github.com/astor194/dataengineering/blob/master/flink_cdc.md
flink_cdc.mdhttps://github.com/astor194/dataengineering/blob/master/flink_cdc.md
READMEhttps://github.com/astor194/dataengineering
https://github.com/astor194/dataengineering#a-very-long-never-ending-learning-around-data-engineering--machine-learning
https://github.com/astor194/dataengineering/blob/master/sketchnotes/background.jpeg
https://github.com/astor194/dataengineering#interesting-reads
How to choose a Distributed Databasehttps://github.com/abhishek-ch/around-dataengineering/blob/master/docs/how_to_choose_db.md
Cockroach DB Architecturehttps://www.linkedin.com/posts/iamabhishekchoudhary_cockroachdbarchitecture-activity-6834117247510855680-VeCF
CockroachDB: The Resilient Geo-Distributed SQL Databasehttps://iswade.github.io/translate/crdb/crdb_paper/
Amundsen Reviewhttps://github.com/abhishek-ch/around-dataengineering/blob/master/docs/amundsen_review.md
Deep Dive - Foundation DBhttps://github.com/abhishek-ch/around-dataengineering/blob/master/docs/foundationdb.md
The What, Why, and When of Single-Table Design with DynamoDBhttps://www.alexdebrie.com/posts/dynamodb-single-table/
How To Manage And Monitor Apache Spark On Kuberneteshttps://www.lightbend.com/blog/how-to-manage-monitor-spark-on-kubernetes-deep-dive-kubernetes-operator-for-spark
Git is hard: screwing up is easy, and figuring out how to fix your mistakes is fucking impossiblehttps://ohshitgit.com/
8 Practical Use Cases of Change Data Capturehttps://medium.com/event-driven-utopia/8-practical-use-cases-of-change-data-capture-8f059da4c3b7
Apache Iceberg- Linkshttps://github.com/abhishek-ch/around-dataengineering/blob/master/docs/apache_iceberg_read.md
Kubernetes Port Forwarding Managerhttps://www.linkedin.com/posts/iamabhishekchoudhary_kubernetes-kuberforwarder-dataengineering-activity-6821345311655555072-aEqx
Querying Parquet with Precision using DuckDB - Much faster compared to Pandashttps://www.linkedin.com/posts/iamabhishekchoudhary_querying-parquet-with-precision-using-duckdb-activity-6821024627578466304-TgRS
What is Apache Pinot - Usecases & Architecturehttps://www.linkedin.com/posts/iamabhishekchoudhary_building-latency-sensitive-user-facing-analytics-activity-6818810335286370304-gZAQ
Change Data Streaming Patterns in Distributedsystemshttps://www.linkedin.com/posts/iamabhishekchoudhary_cdcpatterns-activity-6828573352563654656-CDS1
Cuckoo Hashing - An alternative to chaining and linear probing for collision handling https://www.linkedin.com/posts/iamabhishekchoudhary_cuckoo-hashing-activity-6832911622126784512-3rZB
Riak Databasehttps://github.com/abhishek-ch/around-dataengineering/blob/master/docs/riakdb.md
Database Indexinghttps://www.linkedin.com/posts/iamabhishekchoudhary_databaseindexes-activity-6844509295476924416-Jzto
https://github.com/astor194/dataengineering#weekly-digest
How #dataengineering get complicated over timehttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-realtime-streamprocessing-activity-6808383006567383040-xY-g
What is eBPF - Sandboxing Programs inside #linux Kernelhttps://www.linkedin.com/posts/iamabhishekchoudhary_facebook-google-isovalent-microsoft-and-activity-6831817260462608384-p6QN0
Absolute Basic Explanation of SSTable & Log Structured Merge Trees - Sorted String Table & Faster Random Writeshttps://www.linkedin.com/posts/iamabhishekchoudhary_datastructures-cassandra-dataengineering-activity-6833629108484784128-u-bN
Future-of-the-modern-data-stack-by-dbthttps://blog.getdbt.com/future-of-the-modern-data-stack/
Continual.ai: The future of the modern data stackhttps://continual.ai/post/the-future-of-the-modern-data-stack
https://github.com/astor194/dataengineering#paper
https://github.com/astor194/dataengineering#distributed-system
https://github.com/astor194/dataengineering#crazy
The Snowflake Paper - Core idea is to build an enterprise-ready #datawarehouse solution for the #cloudhttps://www.linkedin.com/posts/iamabhishekchoudhary_snowflakepaper-activity-6752909762079326208-o1cC
Most important points around Distributed #dataengineering Platformhttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-programminglanguages-distributedsystem-activity-6742777508380360704-qqvV
Fundamental of #distributedsystems Scaling - Avoiding Co-ordination πŸŽŠβ™¨οΈπŸ”†https://www.linkedin.com/posts/iamabhishekchoudhary_marcs-blog-activity-6758678779893231616-OX_h
Technical Debt in #dataengineering #softwareengineering πŸ”•πŸ’‘πŸ”•https://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-softwareengineering-programmers-activity-6756946484488400896-2uQx
Paper on Wander Join: Online Aggregation via Random Walks πŸ“ƒπŸ’­πŸ“‘ Join problemhttps://www.linkedin.com/posts/iamabhishekchoudhary_wanderjoinonline-aggregation-problem-activity-6756537519698956288-jTdS
The Delta Lake Paper - High-Performance ACID Table Storage πŸ“‹πŸ’‘πŸ“‹https://www.linkedin.com/posts/iamabhishekchoudhary_thedeltalakepaper-activity-6755858637895335936-kwaT
Dynamo - AWS Highly Available Key-value Store #distributedsystem πŸ’¬πŸ’‘πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_dynamodbkv-activity-6755499276400431104-DofI
An Efficient and Syntactically Idiomatic Approach to Management of Streams and Tables, A Single SQL for all πŸ’‘πŸ“©πŸ“©https://www.linkedin.com/posts/iamabhishekchoudhary_sqlforall-activity-6755119918598955008-WKY5
Secure & Robust Machine Learning in #healthcare πŸ’ŠπŸ§ͺπŸ₯³https://www.linkedin.com/posts/iamabhishekchoudhary_robustmlinhealthcare-activity-6754772406499377152-giZp
Progress in Medical Science using #deeplearning πŸ’ŠπŸ’‘πŸ’‰https://www.linkedin.com/posts/iamabhishekchoudhary_deeplearningcnnmedical-activity-6754374045988380672-exQZ
The Amazon Redshift Paper - A fast, fully managed, petabyte-scale data warehouse solution that makes it simple and cost-effective to efficiently analyze large volumes of data using existing #businessintelligence tools πŸ“‚πŸ“°πŸ’­https://www.linkedin.com/posts/iamabhishekchoudhary_redhistinitialpaper-activity-6753300440881926145-qZY6
Advancing #drugdiscovery via Artificial Intelligence πŸ’ŠπŸ₯πŸ₯https://www.linkedin.com/posts/iamabhishekchoudhary_drugdiscoveryai-activity-6751143165618651136-6EX8
Apache Calcite is a dynamic data management framework πŸŽ‰πŸ“šπŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_apachecalcite-activity-6751114208450015232-67Wn
Lakehouse - A Paper on new Generation of #datawarehouse technology πŸ’‘πŸ”ŽπŸ’‘https://www.linkedin.com/posts/iamabhishekchoudhary_datawarehouse-apacheparquet-machinelearning-activity-6747813594395668480-hepU
Calvin: Fast Distributed Transactions for Partitioned Database Systems πŸ“πŸ“https://www.linkedin.com/posts/iamabhishekchoudhary_calvin-fast-distributed-transactions-activity-6760230219019354113-b-Bi
Presto or Trino - #SQL on Everything ( The Design, Motivation & Performance) #presto πŸ’­πŸŽŠπŸ’‘https://www.linkedin.com/posts/iamabhishekchoudhary_trinoprestosqloneverything-activity-6759416018025680896-DPfu
Design - Exactly Once Delivery & Transactional Messaging in Apache Kafkahttps://www.linkedin.com/posts/iamabhishekchoudhary_exactlyoncekafka-activity-6762359693802299392-Gch3
Apache Kafka Paper : Distributed Messaging System for Log Processinghttps://www.linkedin.com/posts/iamabhishekchoudhary_apachekafkapaper-activity-6765285530247684097-ufXy
Paper: Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large sizehttps://www.linkedin.com/posts/iamabhishekchoudhary_bigdatagoogle-activity-6765973408174923776-jrJz
Paper: Ground is an open-source data context service, a system to manage all the information that informs the use of datahttps://www.linkedin.com/posts/iamabhishekchoudhary_metadatadatacontextservice-activity-6767048256766697472-0A8Q
Azure Data Lake Store(ADLS) is a fully-managed, elastic, scalable, and secure file system that supports #hadoop distributed file system (HDFS) and Cosmos semanticshttps://www.linkedin.com/posts/iamabhishekchoudhary_azure-data-lake-store-activity-6776108273075589120-hqWr
An LFU (Least Frequently Used) Cache eviction algorithm of O(1) Runtime complexityhttps://www.linkedin.com/posts/iamabhishekchoudhary_lfu-cache-eviction-scheme-activity-6780125496920346624-BfKP
The Berkeley View on Cloud Computing - Paperhttps://www.linkedin.com/posts/iamabhishekchoudhary_serverlesscomputing-activity-6773574741639917570-uEB5
The Google File System - The Paper πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_the-google-file-system-paper-activity-6778681429632049152-EF03
Paper: Report on Distributed Deep Learning on Data Systems πŸ“‚https://www.linkedin.com/posts/iamabhishekchoudhary_tech-report-of-distributed-deep-learning-activity-6787729738124341249-fFG8
Crystal: A Unified Cache Storage System for Analytical Databaseshttps://www.linkedin.com/posts/iamabhishekchoudhary_crystal-semantic-caching-activity-6834495561652076544-UGVO
VoltDBhttps://www.linkedin.com/feed/update/urn:li:activity:6846055501105553408/
Magnet - Apache Spark Shuffle mechanism to handle petabytes of daily shuffled data and clusters with thousands of nodeshttps://www.linkedin.com/posts/iamabhishekchoudhary_magnet-sharkshuffleservice-activity-6851903731227799552-EmLz
https://github.com/astor194/dataengineering#set-2
Paper: Real-time Data Infrastructure @ Uberhttps://www.linkedin.com/posts/iamabhishekchoudhary_real-time-data-infrastructure-activity-6792384716063469568-UKc-
Paper: DBLog, A Watermark Based Change-Data-Capture Framework by Netflixhttps://www.linkedin.com/posts/iamabhishekchoudhary_change-data-capture-dblog-activity-6793492331967365120-517C
Paper: Large Scale Distributed Systems Tracing Infrastructurehttps://www.linkedin.com/posts/iamabhishekchoudhary_dapper-a-large-scale-distributed-tracing-infrastructure-activity-6796076018366070784-tHMZ
Paper:Paxos vs Raft: Distributed Consensus πŸ“‹https://www.linkedin.com/posts/iamabhishekchoudhary_paxosvsraft-activity-6798953621909250048-4kgw
Paper: Sorting in a #distributedsystem πŸ“‹https://www.linkedin.com/posts/iamabhishekchoudhary_distributedsystemsorting-activity-6800374901174792192-6C5O
Paper: A large scale analysis of hundreds of in-memory cache clustershttps://www.linkedin.com/posts/iamabhishekchoudhary_a-large-scale-analysis-in-memory-cache-clusters-activity-6802207087511298048-piMh
Design & Architecture of Amazon Timestream - Streaming at Scalehttps://www.linkedin.com/posts/iamabhishekchoudhary_sql-data-dataengineering-activity-6805811689687281664-4edZ
Distributed System Synchronizationhttps://www.linkedin.com/posts/iamabhishekchoudhary_distributedsystemclocksynchronization-activity-6808035719366373376-T_GK
Paper: Consistent hashing - Resizing cluster or Load in a #distributedsystems with a simple concepthttps://www.linkedin.com/posts/iamabhishekchoudhary_consistenthashing-activity-6807248364808282112-2ef2
Deep Dive - Foundation DB (unbundled database, OLTP, strict serializability, multi-version concurrency control, optimistic concurrency control, simulation testing)https://www.linkedin.com/posts/iamabhishekchoudhary_foundationdb-opensource-database-activity-6828300478351237120-ENDF
Distributed Database - ZippyDB is the largest strongly consistent, geographically distributed key-value store at Facebook Databasehttps://www.linkedin.com/posts/iamabhishekchoudhary_database-rocksdb-zookeeper-activity-6830432000072048640-y8nw
BigData Metadata Management Systemhttps://github.com/abhishek-ch/around-dataengineering/blob/master/docs/Metadata_management.md
Machine Learning for Database Optimizationshttps://www.linkedin.com/posts/iamabhishekchoudhary_databases-machinelearning-dataengineering-activity-6841287258293846016-QRVZ
SingleStore - A Distributed Database Management System. It's really more than a Databasehttps://www.linkedin.com/posts/iamabhishekchoudhary_singlestore-database-activity-6842396205964681216-3SLY
ArrowSAM, in-memory genomics SAM format based on Apache Arrowhttps://www.linkedin.com/posts/iamabhishekchoudhary_arrowsam-apache-arrow-based-genomics-sam-activity-6843895456158814208-K86l
Realtime Data Processing FB - Deep Dive on #streamprocessinghttps://www.linkedin.com/posts/iamabhishekchoudhary_realtimedataprocessingatfacebook-activity-6849698562062786560-UhTQ
https://github.com/astor194/dataengineering#na
Glow is an open-source toolkit for working with genomic data at biobank-scale and beyond using #apachespark & #deltalake πŸ’ŠπŸŽ‰πŸ’‰https://www.linkedin.com/posts/iamabhishekchoudhary_apachespark-deltalake-bioinformatics-activity-6758414696254119936-RCz5
ExPASy - Databases and software tools in proteomics, #genomics, phylogeny, systems biology, evolution, population genetics, and transcriptomics πŸ’‘πŸ’ŠπŸ’Šhttps://www.linkedin.com/posts/iamabhishekchoudhary_genomics-bioinformatics-expasy-activity-6750752435607793664-4M1Q
https://github.com/astor194/dataengineering#the-data-engineering
https://github.com/astor194/dataengineering#level-0
Getting started with #dataengineering Volume 6 πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-datapipelines-dag-activity-6770259833363980288-7biE
Getting started with Dataengineering Volume 5 πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-infrastructure-docker-activity-6763429524764975104-lVvy
Getting started with Data Engineering, volume 4 πŸŽ‰πŸ’‘https://github.com/astor194/dataengineering/blob/master/sketchnotes/Getting_started_with_de_vol4.PNG
Getting started with Data Engineering, volume 3 πŸŽ‰πŸ’‘https://github.com/astor194/dataengineering/blob/master/sketchnotes/Getting_started_with_de_vol3.png
Getting started with Data Engineering, volume 2 πŸŽ‰πŸ’‘https://github.com/astor194/dataengineering/blob/master/sketchnotes/Getting_started_with_de_vol2.png
Getting started with Data Engineering, volume 1 πŸŽ‰πŸ’‘https://github.com/astor194/dataengineering/blob/master/sketchnotes/gettingstart_dataengg.png
Getting started with #dataengineering from basicshttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-mapreduce-docker-activity-6803350978432180224-GnWW
Apache Airflow 2.0https://github.com/astor194/dataengineering/blob/master/sketchnotes/Airflow_2_0.jpg
Some Interesting essentials while learning Apache Airflowhttps://github.com/astor194/dataengineering/blob/master/sketchnotes/airflow_checklist.png
Dagster Release 0.10.0 - Everything about Exactly-once, Fault-Tolerant Scheduling - Extremely Important Release πŸŽ‰πŸŽ‰πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_dagster-machinelearning-kubernetes-activity-6757931107041255424-KAG1
#getdbt or Data Build Tools interface across all major Data Workflow Management Platform πŸ’―βœ¨πŸ”₯https://www.linkedin.com/posts/iamabhishekchoudhary_getdbt-apacheairflow-dagster-activity-6750389785275240448-N8t_
Apache Superset - An #opensource Fully Featured Business Intelligence Application 🎊🎊🎊https://www.linkedin.com/posts/iamabhishekchoudhary_apache-superset-10-is-out-activity-6759044550578229248-rerO
The Hop Orchestration Platform, or Apache #Hop (Incubating), aims to facilitate all aspects of data and metadata orchestration πŸ’―πŸ’‘β­https://www.linkedin.com/posts/iamabhishekchoudhary_hop-apachespark-apacheflink-activity-6761641585437396993-ydHg
Apache Iceberg Partitioning is way better than Hive ! Hidden Partitioning makes everything easier! πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_apacheiceberg-dataengineering-bigdata-activity-6764569890650238976-0skJ
Trino aka #prestosql is different from Apache Spark SQL - Exclusively designed for Distributed SQL πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_prestosql-bigdata-apachespark-activity-6764906382325010432-qg00
Apache Spark is NOT a Map but an MPP/MPI Enginehttps://www.linkedin.com/posts/iamabhishekchoudhary_apachespark-mapreduce-dataengineering-activity-6765550560524476416-2c7f
Apache Hudi - Design Principleshttps://www.linkedin.com/posts/iamabhishekchoudhary_apachehudi-hadoop-streamprocessing-activity-6767445258792947712-m9c2
OpenTelemetry specification V1.0https://www.linkedin.com/posts/iamabhishekchoudhary_opentelemetry-specification-v100-tracing-activity-6767783538310873089-N9q5
Everything Around PySpark Pandas UDF πŸ“–https://www.linkedin.com/posts/iamabhishekchoudhary_revisitingpandasudf-activity-6775378803897221120-rS3z
Important skill-set of a Dataengineer - Reduce Costhttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineer-cloud-dataengineering-activity-6783023471485091840-2QfG
Everything on PyFlink - Python with Apache Flinkhttps://www.linkedin.com/posts/iamabhishekchoudhary_apacheflink-dataengineering-python-activity-6785834728302948353-0I0R
Delta Lake Cheat Sheethttps://www.linkedin.com/posts/iamabhishekchoudhary_deltalakecheatsheet-activity-6787337208899678208-LSG3
Dataengineering schedule breakdown, a very flexible estimatehttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-distributedsystems-mlops-activity-6791336683653648384-RY3_
Parquet - Introduction & Design, An OpenSource File Formathttps://www.linkedin.com/posts/iamabhishekchoudhary_apacheparquet-protocolbuffers-dataengineering-activity-6823244946166857729-0CZf
SQL - Avoiding Antipatternshttps://www.linkedin.com/posts/iamabhishekchoudhary_sqlantipattern-activity-6826491257410396160-xX5P
Explaining Apache Kafka - In children's book formathttps://www.linkedin.com/posts/iamabhishekchoudhary_apachekafka-streamprocessing-dataengineering-activity-6811538408763121664-5qpL
The Perfect #dataengineering: Top INVALID Reasons behind #datapipelines failureshttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-datapipelines-data-activity-6839946812015603712-FnZq
https://github.com/astor194/dataengineering#level-1
DataEngg Skills to work with DataSciencehttps://github.com/astor194/dataengineering/blob/master/sketchnotes/DE_skills_work_with_DS.jpg
Data Quality, A necessity for Data Driven Projectshttps://github.com/astor194/dataengineering/blob/master/sketchnotes/arond_dq.png
Essential Cloud Skills for Data Engineeringhttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-machinelearnig-aws-activity-6758329553036345346-b-40
Open Source Technologies in Data Engineeringhttps://github.com/astor194/dataengineering/blob/master/sketchnotes/DE_OS.jpg
Kubernetes Fundamentals Required as a Data Engineerhttps://github.com/astor194/dataengineering/blob/master/sketchnotes/kubernetes_fundamentals.png
Apache Superset, OSS Business Intelligence for 2021https://www.linkedin.com/posts/iamabhishekchoudhary_apache-superset-10-is-out-activity-6759044550578229248-rerO
#apachekafka as a Database - Summary on both the sides , Arguments, Trade-offs & exceptional πŸ’¬ quotes β³πŸ’‘β³https://www.linkedin.com/posts/iamabhishekchoudhary_kafka-as-a-database-yes-or-no-a-summary-activity-6757228852923158528-qVBc
Processing Guarantees in #apachekafka πŸ’―πŸ”†πŸŽ‰ - The best resourcehttps://www.linkedin.com/posts/iamabhishekchoudhary_processing-guarantees-in-kafka-activity-6756149463359791104-sJcD
Change Data Analysis with Debezium and Apache Pinot πŸŽ‰πŸ’‘πŸšΏhttps://www.linkedin.com/posts/iamabhishekchoudhary_apachepinot-debezium-eventsourcing-activity-6754311102718382080-gWL1
Optimizing Apache Kafka Producers & Consumers πŸŽŠπŸ“ˆπŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_apachekafka-strimzi-streamprocessing-activity-6753977207800037376-kNWi
Redpanda -A NON-JVM Streaming Platform for mission critical workloads πŸ’‘πŸŽ‰πŸ”†https://www.linkedin.com/posts/iamabhishekchoudhary_redpanda-activity-6749640173656559616-Ol6C
Apache Hudi - Turn Batch Jobs to Incremental Model | Complete file management on a Data Lakehttps://www.linkedin.com/posts/iamabhishekchoudhary_building-a-large-scale-transactional-data-activity-6760855681277980672-kiZF
Apache Iceberg - an open table format for huge analytic datasetshttps://www.linkedin.com/posts/iamabhishekchoudhary_apache-iceberg-activity-6760465145954131968-v0tp
Ballista - Distributed computing platform built primarily on Rust and powered by Apache Arrowhttps://www.linkedin.com/posts/iamabhishekchoudhary_ballista-a-distributed-compute-platform-activity-6763021778580246528-Zwj8
ZooKeeper, a distributed, open-source coordination service for distributed applicationshttps://www.linkedin.com/posts/iamabhishekchoudhary_zookeeperpaper-activity-6764214157366620160-84rM
Apache Iceberg - Partition Evolution, its simple but its so amazinghttps://www.linkedin.com/posts/iamabhishekchoudhary_apacheiceberg-dataengineering-datascience-activity-6766299661784403968-T_-i
ApacheKafka without ZooKeeper Sneak PeakπŸ”Žhttps://www.linkedin.com/posts/iamabhishekchoudhary_apachekafka-distributedsystems-zookeeper-activity-6782957154392432640-3s5r
Why Data Discovery is important for Data Engineeringhttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-bigdata-datagovernance-activity-6797526472681771008-C-XM
Queue vs Log - Event driven Architecturehttps://www.linkedin.com/posts/iamabhishekchoudhary_eventsourcing-distributedsystems-bigdata-activity-6798554566800523264-xa88
https://github.com/astor194/dataengineering#level-11
Multiple criteria search at scale with Apache Pinot & Theta Sketcheshttps://www.linkedin.com/posts/iamabhishekchoudhary_solving-for-the-cardinality-of-set-intersection-activity-6789846326893969408-4exe
VM vs Containers - Similar but Differenthttps://www.linkedin.com/posts/iamabhishekchoudhary_docker-dataengineering-bigdata-activity-6790586642504724480-Zq0e
State of Trino aka PrestoSQLhttps://www.linkedin.com/posts/iamabhishekchoudhary_trino-aka-prestosql-state-activity-6790920739471069184-Ipcq
ETL is an extremely important component for any modern businesshttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-datawarehouse-bigdata-activity-6789114504010625024-V-lz
Top 5 ways to complicate a #dataengineering pipeline/application πŸ’₯https://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-functionalprogramming-bigdata-activity-6802551801179631616-u6a0
Leader election is commonly used aka Master/Namenode/Leader/Driverhttps://www.linkedin.com/posts/iamabhishekchoudhary_leaderelection-activity-6802979035270938624-VYYF
Dagster vs Airflow - A comparisonhttps://www.linkedin.com/posts/iamabhishekchoudhary_moving-past-airflow-why-dagster-is-the-next-generation-activity-6800808471768969216-__wj
About Single Source of Truth in DataEngineeringhttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-machinelearning-dataanalytics-activity-6801541986638733312-81gM
Change Data Capture for Distributed Databaseshttps://www.linkedin.com/posts/iamabhishekchoudhary_change-data-capture-for-distributed-databases-activity-6821429243944206336-HiJL
Deep Dive on Why Apache Iceberg for Change Data Capture, using Apache Flink πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_apacheiceberg-deltalake-realtime-activity-6813803694627336192-W7M7
OpenMetadata is an Open Standard for Metadata. A Single place to Discover, Collaborate, and Get your data righthttps://www.linkedin.com/posts/iamabhishekchoudhary_metadata-datamesh-amundsen-activity-6833223767699845120-8U5h
About Lakehousehttps://www.linkedin.com/posts/iamabhishekchoudhary_datawarehouse-lakehouse-machinelearning-activity-6839506317648941056-d23r
https://github.com/astor194/dataengineering#gyaan
A Data Engineering Story - The Beginninghttps://github.com/abhishek-ch/around-dataengineering/blob/master/docs/blog1/index.md
Data Engineering - More towards Data Science or Data Analytics or ...https://github.com/abhishek-ch/around-dataengineering/tree/blog2
Data Engineering Interview Patternshttps://github.com/astor194/dataengineering/blob/master/sketchnotes/DE_Interview.jpg
Basic Checklists while learning Apache Sparkhttps://github.com/astor194/dataengineering/blob/master/sketchnotes/spark_checklist.png
#apachespark for Distributed Analytics or #businessinteligence Platform - Worth or not ?https://www.linkedin.com/posts/iamabhishekchoudhary_apachespark-businessinteligence-analytics-activity-6754429988021387264-kA7V
Apache Beam for Search: An Introduction & Addressing the challenge of the Time Problem πŸ”πŸ’‘πŸ”’https://www.linkedin.com/posts/iamabhishekchoudhary_apachebeam-apachekafka-dataengineering-activity-6753591516708573184-1-p6
Nextflow is a Workflow Manager exclusively for #bioinformatics πŸ©ΉπŸ’ŠπŸ©Ήhttps://www.linkedin.com/posts/iamabhishekchoudhary_bioinformatics-distributedsystems-kubernetes-activity-6749644503558238209-ufTg
#apachespark Project Zen Update - Making PySpark Better πŸ’‘πŸ”—πŸ’‘https://www.linkedin.com/posts/iamabhishekchoudhary_project-zen-improving-apache-spark-for-python-activity-6748904711069474816-3iQN
Design - Exactly Once Delivery & Transactional Messaging in #apachekafka πŸŽŠπŸ“‹πŸŽŠhttps://www.linkedin.com/posts/iamabhishekchoudhary_exactlyoncekafka-activity-6762359693802299392-Gch3
underrated but important skill of a Data Engineerhttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineer-datascience-dataengineering-activity-6775777891335647232-6573
Fallacies of Distributed Systemshttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-distributedsystems-cloud-activity-6776439768591998976-aE6h
As a Data Engineer, some Essentials I did which really helped Data Scientists and the Teamhttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineer-datascientists-kubernetes-activity-6779784194651504640-zIUP
A very normal Data Engineering work πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-python-kubernetes-activity-6772461203500384256-yrAH
What can go wrong in Distributed Data Systemshttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineers-dataengineering-machinelearning-activity-6771387632715935744-wZCk
Architect and build an #machinelearning use case end to end using Amazon SageMaker πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_architect-and-build-the-full-machine-learning-activity-6775727947325214721-iYAW
Around Data Discovery or Metadata Management Platformshttps://www.linkedin.com/posts/iamabhishekchoudhary_datadiscovery-metadatamanagement-datascientists-activity-6777238261258682368-fkGa
Amazon S3 Object Lambda - Provide Different Views of Data to Multiple Applicationshttps://www.linkedin.com/posts/iamabhishekchoudhary_introducing-amazon-s3-object-lambda-use-activity-6778582018067378176-fzvy
Full Stack Data Engineerhttps://www.linkedin.com/posts/iamabhishekchoudhary_hadoop-apachespark-kubernetes-activity-6781152322971074560-VWvK
Data cleaning is Hard but whyhttps://www.linkedin.com/posts/iamabhishekchoudhary_whys-it-hard-to-teach-data-cleaning-activity-6782684736637693953-JiLg
Most exciting things about #dataengineeringhttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-distributedsystems-machinelearning-activity-6783666378680401920-E23H
The real impact of Disks on #rocksdb State Backend in Apache Flinkhttps://www.linkedin.com/posts/iamabhishekchoudhary_the-impact-of-disks-on-rocksdb-state-backend-activity-6784115938338906112-FTVY
Tips for Distributed System High Availabilityhttps://www.linkedin.com/posts/iamabhishekchoudhary_tips-for-high-availability-activity-6785942817693855744-8KFF
interesting way of collaboration between a Dataengineer & Datascientishttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineer-datascientist-datascience-activity-6786211019640369152-QrFF
Building DistributedLog: High-performance replicated log service πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_distributedsystems-dataengineering-bigdata-activity-6786283090441506816-kCob
Whiz: Data Analytics Execution Framework based on Intermediate Datahttps://www.linkedin.com/posts/iamabhishekchoudhary_whiz-data-driven-analytics-execution-activity-6788003925300707328-2Yuy
Adding unlimited Nodes in a #dataengineering platform will eventually drophttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-distributedsystems-bigdata-activity-6788418825047080960-iTiv
A typical Data Engineering Pipelinehttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-data-bigdata-activity-6800075858917773312-W_Lc
'Log' is a fundamental component of a Data Engineering Ecosystemhttps://www.linkedin.com/posts/iamabhishekchoudhary_apacheiceberg-deltalake-dataengineering-activity-6804701361796636672-MOFm
Flink CDChttps://github.com/abhishek-ch/around-dataengineering/blob/master/flink_cdc.md
Readings Around Databaseshttps://github.com/abhishek-ch/around-dataengineering/blob/master/docs/Reading_ardound_database.md
Code Review Best Practice, bcz Developers, hate code reviewshttps://www.linkedin.com/posts/iamabhishekchoudhary_codereviewbestpractice-activity-6812358358906019840-Qfw5
Important Performance Criteria to measure DataEngineering Systemshttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-distributedsystems-datastructures-activity-6848612433121157120-wrNi
https://github.com/astor194/dataengineering#infrastructure
SQL Database on Kubernetes - Best Practiceshttps://www.linkedin.com/posts/iamabhishekchoudhary_sql-database-on-kubernetes-considerations-activity-6755432993130643457-Zts2
Devtron - An Open Source DevOps on Kubernetes, written in Go πŸ₯‡πŸŽπŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_devtron-multicloud-infrastructure-activity-6749978071970975744-PcPs
Most Popular #opensource BI & Data Analytics Platforms πŸŽŠπŸ’‘πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_opensource-dataengineering-datascience-activity-6749258103369322496-hc5C
datapipelines Dataframe APi is now available with #apachebeam πŸ’―πŸ”₯πŸ’―https://www.linkedin.com/posts/iamabhishekchoudhary_datapipelines-apachebeam-python-activity-6748167921433948160-oWfT
Disaster Recovery for Multi-Region Apache Kafka & Data Consumption using #apacheflink πŸ”…πŸŽ‰πŸ”…https://www.linkedin.com/posts/iamabhishekchoudhary_disaster-recovery-for-multi-region-kafka-activity-6747422409218940928-70Rf
Kubernetes Api Structure πŸ’―βœ”οΈπŸ’―https://www.linkedin.com/posts/iamabhishekchoudhary_kubernetes-dataengineering-machinelearning-activity-6759074145817944064-dHIl
Architecting a Kubernetes Infrastructure πŸ’―https://www.linkedin.com/posts/iamabhishekchoudhary_a-deep-dive-into-architecting-a-kubernetes-activity-6761257898438918144-N74g
Exploring Kubernetes Operator Pattern πŸ’‘https://www.linkedin.com/posts/iamabhishekchoudhary_exploring-kubernetes-operator-pattern-activity-6762000800911814656-Jre7
Docker is an interal part of Data Engineering ML pipeline & that makes security πŸ” extremely essentialhttps://www.linkedin.com/posts/iamabhishekchoudhary_docker-security-activity-6763766659426660353-_MIa
Rack awareness for #apachekafka Streams Proposal πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_apachekafka-kafkastreams-dataengineering-activity-6779000842264592384-NA4J
Dolt is Git for Data 🎊https://www.linkedin.com/posts/iamabhishekchoudhary_sql-bigdata-datascience-activity-6774623291329060864-QKYb
Toward Better Data Culture From First Principles by Ubehttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineering-dataengineer-bigdata-activity-6780824810315403264-e0hC
Fast and Reliable Schema-Agnostic Log Analytics Platform by Uberhttps://www.linkedin.com/posts/iamabhishekchoudhary_elasticsearch-elk-dataengineering-activity-6783364868515934208-jhj6
Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive SystemsπŸ“‹https://www.linkedin.com/posts/iamabhishekchoudhary_simple-testing-can-prevent-most-critical-failures-activity-6784411707830779904-ds3_
Diving Deep on S3 Consistency - Insightfulhttps://www.linkedin.com/posts/iamabhishekchoudhary_diving-deep-on-s3-consistency-activity-6793185152844513280-k48V
Ray- General Purpose ML Infrastructurehttps://github.com/abhishek-ch/around-dataengineering/blob/master/docs/ray_distributed_ml.md
Kubernetes Hardening Guide by National Security Agencyhttps://www.linkedin.com/posts/iamabhishekchoudhary_kubernetes-hardening-guide-activity-6831118268779020288-KQfU
Everything around Load Balancerhttps://www.linkedin.com/posts/iamabhishekchoudhary_distributedsystems-datainfrastructure-loadbalancing-activity-6840255837957607424-oOmk
Data Lakehouse - Is it really the end of Datawarehousehttps://www.linkedin.com/posts/iamabhishekchoudhary_datawarehouse-lakehouse-machinelearning-activity-6839506317648941056-d23r
Real-Time Exactly-Once Processing with Apache Flink, Kafka, and Pinot @uberhttps://www.linkedin.com/posts/iamabhishekchoudhary_distributedsystems-apacheflink-apachekafka-activity-6850759002620522496-zq8w
WTH is Kubernetes Operator? - An Introductionhttps://www.linkedin.com/posts/iamabhishekchoudhary_k8s-operator-introduction-activity-6850331132089679872-_sVt
https://github.com/astor194/dataengineering#sql
Advanced SQL - Reference CS 564 Database Management Systemshttps://www.linkedin.com/posts/iamabhishekchoudhary_advanced-sql-2-activity-6845598540790599680-gTgH
SQL and Advanced SQL - An assethttps://www.linkedin.com/posts/iamabhishekchoudhary_advancedsql-activity-6844892160798584832-gBTl
Database Indexing - Almost Everythinghttps://www.linkedin.com/posts/iamabhishekchoudhary_databaseindexes-activity-6844509295476924416-Jzto
Tuning SQL queries - Tips for writing efficient & faster Querieshttps://www.linkedin.com/posts/iamabhishekchoudhary_query-optimization-techniques-sql-activity-6844123194836742144-aptY
Database Schema Design - Schema Design is a Complicated Necessityhttps://www.linkedin.com/posts/iamabhishekchoudhary_database-schema-design-activity-6843509027025022976-qbRF
SQL Query Processing Plan - Basicshttps://www.linkedin.com/posts/iamabhishekchoudhary_queryprocessingplan-activity-6843139272497774592-10sT
Revisiting SQL Basics - The beginning of Data Science & Data Engineeringhttps://www.linkedin.com/posts/iamabhishekchoudhary_basic-sql-activity-6842758586142142464-kJ_U
Distributed Advanced Queries - Presto/Trinohttps://www.linkedin.com/posts/iamabhishekchoudhary_advanced-sql-distributed-query-engine-prestotrino-activity-6845952140645593088-r6BJ
SQL Notes For Professionals, 100+ pageshttps://www.linkedin.com/posts/iamabhishekchoudhary_sqlnotes4professional-activity-6846303034914471936-q1T9
Table Partitioninghttps://www.linkedin.com/posts/iamabhishekchoudhary_database-dataengineering-sql-activity-6846689189979844609-yXQS
SQL complex Queries - Nested Queries & Aggregationhttps://www.linkedin.com/posts/iamabhishekchoudhary_nested-queries-and-aggregation-activity-6847059870416474112-tSkY
Gossip Protocols - Designed for Data Consistency & Fault-Tolerancehttps://www.linkedin.com/posts/iamabhishekchoudhary_p2p-gossip-protocol-activity-6847021932785786880-A2Oj
Table Partitioning - An Important Concepthttps://www.linkedin.com/posts/iamabhishekchoudhary_database-dataengineering-sql-activity-6846689189979844609-yXQS
Database Concurrency Control : 2 Phase Lockinghttps://www.linkedin.com/posts/iamabhishekchoudhary_sql-database-dataengineering-activity-6847472978335195136-qqaC
Database Entity Relationship Modelhttps://www.linkedin.com/posts/iamabhishekchoudhary_the-entity-relationship-er-model-activity-6848113792358907904-bwRc
SQL Join Fundamentalshttps://www.linkedin.com/posts/iamabhishekchoudhary_sql-joins-activity-6848474520647458816-3ACA
Database Indexinghttps://www.linkedin.com/posts/iamabhishekchoudhary_database-performanceandindexes-activity-6848848899260653569-l1gz
Database Indexing Noteshttps://www.inf.unibz.it/~artale/DB2/handout2.pdf
SQL Injection Introductionhttps://www.linkedin.com/posts/iamabhishekchoudhary_sql-injection-introduction-activity-6849573020533608448-tsWX
SQL Constraints Fundamentalshttps://www.linkedin.com/posts/iamabhishekchoudhary_sql-constraints-activity-6849207436176363520-hiDU
The fundamental of writing SQL queries is different fromhttps://www.linkedin.com/posts/iamabhishekchoudhary_programming-dataengineering-datanalytics-activity-6797939314098520064-hMfd
Building a NoSQL Database using Githttps://www.linkedin.com/posts/iamabhishekchoudhary_gitasnosqldatabase-activity-6820624465861390336-qwpo
Against SQL - An article on What is not good with #sql https://www.linkedin.com/posts/iamabhishekchoudhary_sql-database-data-activity-6819964297423179776-OPr5
Using EXPLAIN for Data Problems - Things beyond SQLhttps://www.linkedin.com/posts/iamabhishekchoudhary_databaseexplain-activity-6819573919389941760-euD5
10 SQL Queries to Blow Your Mind πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_sqlmightiestqueries-activity-6829031645870374912-HW88
Views, Stored Procedures, Functions & Triggers - SQL https://www.linkedin.com/posts/iamabhishekchoudhary_viewstoredproceduretriggersql-activity-6850663245687115776-1V75
SQL Transaction & ACID Propertyhttps://www.linkedin.com/posts/iamabhishekchoudhary_sql-transaction-activity-6851020805803958272-w69d
How to Solve complex SQL querieshttps://www.linkedin.com/posts/iamabhishekchoudhary_sqlcomplexqueriesapproach-activity-6851755823228604416-uoVK
Apache Spark SQL - The Introduction from RDMBS till SparkSQLhttps://www.linkedin.com/posts/iamabhishekchoudhary_sparksqlintroduction-activity-6852476659761782785-MOLf
https://github.com/astor194/dataengineering#machine-learning
https://github.com/astor194/dataengineering#mlops
Machine Learning Workflow πŸ’― https://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-dataengineering-distributedsystems-activity-6754711934441984000-WZ1G
Dummy Notes On Machine Learning Infrastructirehttps://github.com/astor194/dataengineering/blob/master/sketchnotes/ML_Infra_RoughNotes.png
Machine Learning Feature Store πŸ’―https://github.com/astor194/dataengineering/blob/master/sketchnotes/feature_store.jpg
Deploying #machinelearning model in Production is really HARD but #MLOps can fix that.https://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-mlops-datascience-activity-6752596084746436609-r99U
List of #machinelearning & #dataengineering Technologies will be following in 2021 πŸŽ‰πŸ’‘πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-dataengineering-getdbt-activity-6751786250769903616-sEUb
MLOps - ZenML #machinelearning with reproducible pipelines βœ…πŸ’―βœ…https://www.linkedin.com/posts/iamabhishekchoudhary_maiot-iozenml-activity-6748236907542577152-nBOj
Why? Data Versioning is a complicated problem for Dataengineershttps://www.linkedin.com/posts/iamabhishekchoudhary_dataengineers-apacheiceberg-deltalake-activity-6793880325475901440-Lsxx
Explainable AI Cheat Sheethttps://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-mlops-dataengineering-activity-6794228285145452545-YEwQ
https://github.com/astor194/dataengineering#project
Streamlit Healthcare Machine Learning Data Apphttps://github.com/abhishek-ch/streamlit-healthcare-ML-App
Dstack AI - An open-source tool to develop data applications with Python πŸŽ‰πŸ’­πŸŽŠhttps://www.linkedin.com/posts/iamabhishekchoudhary_dstack-machinelearning-businessinteligence-activity-6753247298588852225-CbcA
Adversarial Robustness Toolbox - a Python library for #machinelearning Security πŸ’‘πŸ”ŽπŸ”“https://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-dataengineering-datascience-activity-6752863649414680576-bSK8
Biopython is a set of freely available tools for biological computation written in #Python πŸ’ŠβŒ›οΈπŸ’Šhttps://www.linkedin.com/posts/iamabhishekchoudhary_python-bioinformatics-apachespark-activity-6749263102992269312-SCut
https://github.com/astor194/dataengineering#insightful
Time to Know More about DASKhttps://github.com/astor194/dataengineering/blob/master/sketchnotes/TimeToLearnDask.png
DataEngineering vs Machine Learninghttps://github.com/astor194/dataengineering/blob/master/sketchnotes/DataEngineeringVsMachineLearningengineering.jpg
A good #machinelearning Model is only possible with a good quality of #data. βŒ›οΈhttps://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-data-datascience-activity-6757609262110826496-CmGj
Statistics for #softwareengineer πŸ”₯πŸ’―πŸ”₯https://www.linkedin.com/posts/iamabhishekchoudhary_statsforhackers-activity-6752265241486331904-moUz
Monitoring #machinelearning Applications πŸŽ‰πŸŽ πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-dataengineering-prometheus-activity-6751436872456687616-ZTvW
Dagster is a data orchestrator for machine learning, analytics, and ETL - Officially #machinelearning driven πŸ₯‡πŸ₯‡πŸ₯‡https://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-dagster-apacheairflow-activity-6749618916609314816-YPnW
Short Notes on -Open source #machinelearning Tracking Systemhttps://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-datascience-mlflow-activity-6759484697518096384-Szi9
The best example of Randomness is - #machinelearning model in Production. πŸ”πŸ’­πŸ”Žhttps://www.linkedin.com/posts/iamabhishekchoudhary_machinelearning-infrastructure-datascientists-activity-6759792666403246081-Jq3Q
Flyte is declarative, structured, and highly scalable cloud-native workflow orchestration platform for Distributed Machine Learninghttps://www.linkedin.com/posts/iamabhishekchoudhary_mlops-golang-datalineage-activity-6778218137268625408-WWdm
Tips for Distributed System High Availability πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_tips-for-high-availability-activity-6785942817693855744-8KFF
Building DistributedLog: High-performance replicated log service πŸŽ‰https://www.linkedin.com/posts/iamabhishekchoudhary_distributedsystems-dataengineering-bigdata-activity-6786283090441506816-kCob
How to scale Kubernetes with Assurancehttps://www.linkedin.com/posts/iamabhishekchoudhary_kubernetes-infrastructure-dataengineering-activity-6786619511165501440-pcNS
Apache Calcite - Building Sql Query Processor from Scratch over Lucenehttps://www.linkedin.com/posts/iamabhishekchoudhary_calcite-tutorial-lucene-activity-6833330741460447233-CCfP
https://github.com/astor194/dataengineering#cloud
Readme https://github.com/astor194/dataengineering#readme-ov-file
Please reload this pagehttps://github.com/astor194/dataengineering
Activityhttps://github.com/astor194/dataengineering/activity
0 forkshttps://github.com/astor194/dataengineering/forks
Report repository https://github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fastor194%2Fdataengineering&report=astor194+%28user%29
Releaseshttps://github.com/astor194/dataengineering/releases
Packages 0https://github.com/users/astor194/packages?repo_name=dataengineering
Please reload this pagehttps://github.com/astor194/dataengineering
Contributorshttps://github.com/astor194/dataengineering/graphs/contributors
Please reload this pagehttps://github.com/astor194/dataengineering
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.