René's URL Explorer Experiment

Title: Use the Cloud Client Libraries for Python | Managed Service for Apache Spark | Google Cloud Documentation

Open Graph Title: Use the Cloud Client Libraries for Python | Managed Service for Apache Spark | Google Cloud Documentation

Description: Use the moonrise-replaced27ea3bbc5704b48a440aa7632b63f8cmoonrise-replace client libraries for Python to programmatically create clusters and submit jobs via gRPC APIs.

Open Graph Description: Use the moonrise-replaced27ea3bbc5704b48a440aa7632b63f8cmoonrise-replace client libraries for Python to programmatically create clusters and submit jobs via gRPC APIs.

Opengraph URL: https://docs.cloud.google.com/managed-spark/docs/tutorials/python-library-example

direct link

Domain: docs.cloud.google.com

Hey, it has json ld scripts:

  {
    "@context": "https://schema.org",
    "@type": "Article",
    
    "headline": "Use the Cloud Client Libraries for Python"
  }

  {
    "@context": "https://schema.org",
    "@type": "BreadcrumbList",
    "itemListElement": [{
      "@type": "ListItem",
      "position": 1,
      "name": "Managed Service for Apache Spark",
      "item": "https://docs.cloud.google.com/managed-spark/docs"
    },{
      "@type": "ListItem",
      "position": 2,
      "name": "Use the Cloud Client Libraries for Python",
      "item": "https://docs.cloud.google.com/managed-spark/docs/tutorials/python-library-example"
    }]
  }

google-signin-client-id	721724668570-nbkv1cfusk7kk4eni4pjvepaus73b13t.apps.googleusercontent.com
google-signin-scope	profile email https://www.googleapis.com/auth/developerprofiles https://www.googleapis.com/auth/developerprofiles.award https://www.googleapis.com/auth/devprofiles.full_control.firstparty
og:site_name	Google Cloud Documentation
og:type	website
theme-color	#1a73e8
None	IE=Edge
og:image	https://docs.cloud.google.com/_static/cloud/images/social-icon-google-cloud-1200-630.png
og:image:width	1200
og:image:height	630
og:locale	en
twitter:card	summary_large_image

Links:

Skip to main content	https://docs.cloud.google.com/managed-spark/docs/tutorials/python-library-example#main-content
	https://docs.cloud.google.com/
Documentation	https://docs.cloud.google.com/docs
Get Started with Google Cloud	https://docs.cloud.google.com/docs/get-started
Product List	https://docs.cloud.google.com/docs/product-list
Cloud Customer Care	https://docs.cloud.google.com/support/docs
Agent Platform	https://docs.cloud.google.com/products/gemini-enterprise-agent-platform
Apigee API Management	https://docs.cloud.google.com/apigee
BigQuery	https://docs.cloud.google.com/bigquery
Compute Engine	https://docs.cloud.google.com/products/compute
Cloud CDN	https://docs.cloud.google.com/cdn
Cloud Run	https://docs.cloud.google.com/run
Cloud Storage	https://docs.cloud.google.com/storage
Cloud SQL	https://docs.cloud.google.com/sql
Gemini Enterprise	https://docs.cloud.google.com/gemini/enterprise/docs
Google Kubernetes Engine	https://docs.cloud.google.com/kubernetes-engine
Looker	https://docs.cloud.google.com/looker
Access and resources management	https://docs.cloud.google.com/docs/access-resources
Costs and usage management	https://docs.cloud.google.com/docs/costs-usage
Infrastructure as code	https://docs.cloud.google.com/docs/iac
SDK, languages, frameworks, and tools	https://docs.cloud.google.com/docs/devtools
AI and ML	https://docs.cloud.google.com/docs/ai-ml
Application development	https://docs.cloud.google.com/docs/application-development
Application hosting	https://docs.cloud.google.com/docs/application-hosting
Compute	https://docs.cloud.google.com/docs/compute-area
Data analytics and pipelines	https://docs.cloud.google.com/docs/data
Databases	https://docs.cloud.google.com/docs/databases
Distributed, hybrid, and multicloud	https://docs.cloud.google.com/docs/dhm-cloud
Industry solutions	https://docs.cloud.google.com/docs/industry
Migration	https://docs.cloud.google.com/docs/migration
Networking	https://docs.cloud.google.com/docs/networking
Observability and monitoring	https://docs.cloud.google.com/docs/observability
Security	https://docs.cloud.google.com/docs/security
Storage	https://docs.cloud.google.com/docs/storage
Console	https://console.cloud.google.com/
	https://docs.cloud.google.com/managed-spark/docs
Managed Service for Apache Spark	https://docs.cloud.google.com/managed-spark/docs
Start free	https://console.cloud.google.com/freetrial
Overview	https://docs.cloud.google.com/managed-spark/docs
Guides	https://docs.cloud.google.com/managed-spark/docs/overview/key-concepts
Reference	https://docs.cloud.google.com/managed-spark/docs/reference
Samples	https://docs.cloud.google.com/managed-spark/docs/samples
Resources	https://docs.cloud.google.com/managed-spark/docs/resources
	https://docs.cloud.google.com/
Documentation	https://docs.cloud.google.com/docs
Overview	https://docs.cloud.google.com/managed-spark/docs
Guides	https://docs.cloud.google.com/managed-spark/docs/overview/key-concepts
Reference	https://docs.cloud.google.com/managed-spark/docs/reference
Samples	https://docs.cloud.google.com/managed-spark/docs/samples
Resources	https://docs.cloud.google.com/managed-spark/docs/resources
Console	https://console.cloud.google.com/
Key Concepts	https://docs.cloud.google.com/managed-spark/docs/overview/key-concepts
Overview	https://docs.cloud.google.com/managed-spark/docs/serverless-overview
Managed Service for Apache Spark serverless tiers	https://docs.cloud.google.com/managed-spark/docs/tiers
Managed Service for Apache Spark on clusters	https://docs.cloud.google.com/managed-spark/docs/concepts/clusters-overview
Compare serverless and cluster deployments	https://docs.cloud.google.com/managed-spark/docs/concepts/serverless-spark-compare
Managed Service for Apache Spark on GKE	https://docs.cloud.google.com/managed-spark/docs/concepts/gke-overview
Create a serverless Spark batch workload	https://docs.cloud.google.com/managed-spark/docs/quickstarts/spark-batch
Create a cluster	https://docs.cloud.google.com/managed-spark/docs/quickstarts/create-cluster
Submit a Spark job to a cluster	https://docs.cloud.google.com/managed-spark/docs/guides/submit-job
Use Gemini to develop Spark applications	https://docs.cloud.google.com/managed-spark/docs/guides/gemini-spark
Analyze public datasets with Spark	https://docs.cloud.google.com/managed-spark/docs/tutorials/monte-carlo-methods-with-hadoop-spark
Sentiment analysis with Spark MLlib	https://docs.cloud.google.com/managed-spark/docs/tutorials/bigquery-sparkml
Run a Spark job on Kubernetes	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/quickstarts/gke-quickstart-create-cluster
Use custom containers	https://docs.cloud.google.com/managed-spark/docs/guides/custom-containers
Use GPUs	https://docs.cloud.google.com/managed-spark/docs/guides/gpus-serverless
Use Dynamic Workload Scheduler	https://docs.cloud.google.com/managed-spark/docs/guides/dws-serverless
Network configuration	https://docs.cloud.google.com/managed-spark/docs/concepts/network-serverless
Overview	https://docs.cloud.google.com/managed-spark/docs/concepts/versions/serverless-versions
Spark runtime version 3.0	https://docs.cloud.google.com/managed-spark/docs/concepts/versions/spark-runtime-3.0
Spark runtime version 2.3	https://docs.cloud.google.com/managed-spark/docs/concepts/versions/spark-runtime-2.3
Spark runtime version 2.2	https://docs.cloud.google.com/managed-spark/docs/concepts/versions/spark-runtime-2.2
Spark runtime version 1.2	https://docs.cloud.google.com/managed-spark/docs/concepts/versions/spark-runtime-1.2
Service accounts	https://docs.cloud.google.com/managed-spark/docs/concepts/service-accounts-serverless
Spark properties	https://docs.cloud.google.com/managed-spark/docs/concepts/spark-properties-serverless
Staging bucket	https://docs.cloud.google.com/managed-spark/docs/concepts/staging-bucket-serverless
Create a serverless Spark batch workload	https://docs.cloud.google.com/managed-spark/docs/guides/spark-batch
Create serverless interactive sessions and runtime profiles	https://docs.cloud.google.com/managed-spark/docs/guides/create-serverless-sessions-templates
Use the serverless Spark Connect client	https://docs.cloud.google.com/managed-spark/docs/guides/spark-connect-client
Use JupyterLab for serverless batch and notebook sessions	https://docs.cloud.google.com/managed-spark/docs/guides/jupyterlab-sessions-serverless
Visualize Spark DataFrames in notebooks	https://docs.cloud.google.com/managed-spark/docs/guides/visualize-dataframes-notebooks
Overview	https://docs.cloud.google.com/managed-spark/docs/templates/templates
Cloud Spanner to Cloud Storage	https://docs.cloud.google.com/managed-spark/docs/templates/spanner-to-storage
Cloud Storage to BigQuery	https://docs.cloud.google.com/managed-spark/docs/templates/storage-to-bigquery
Cloud Storage to Cloud Spanner	https://docs.cloud.google.com/managed-spark/docs/templates/storage-to-spanner
Cloud Storage to Cloud Storage	https://docs.cloud.google.com/managed-spark/docs/templates/storage-to-storage
Cloud Storage to JDBC	https://docs.cloud.google.com/managed-spark/docs/templates/storage-to-jdbc
Hive to BigQuery	https://docs.cloud.google.com/managed-spark/docs/templates/hive-to-bigquery
Hive to Cloud Storage	https://docs.cloud.google.com/managed-spark/docs/templates/hive-to-storage
JDBC to BigQuery	https://docs.cloud.google.com/managed-spark/docs/templates/jdbc-to-bigquery
JDBC to Cloud Spanner	https://docs.cloud.google.com/managed-spark/docs/templates/jdbc-to-spanner
JDBC to Cloud Storage	https://docs.cloud.google.com/managed-spark/docs/templates/jdbc-to-storage
JDBC to JDBC	https://docs.cloud.google.com/managed-spark/docs/templates/jdbc-to-jdbc
Pub/Sub to Cloud Storage	https://docs.cloud.google.com/managed-spark/docs/templates/pubsub-to-storage
Create a lakehouse with Spark	https://docs.cloud.google.com/managed-spark/docs/guides/create-lakehouse
Access Workday data with Spark	https://docs.cloud.google.com/managed-spark/docs/guides/access-workday-data-spark
Query federated catalogs with Spark	https://docs.cloud.google.com/managed-spark/docs/guides/query-federated-catalogs
Create an Apache Iceberg table with metadata in Lakehouse runtime catalog	https://docs.cloud.google.com/managed-spark/docs/guides/spark-workloads-with-bigquery-metastore
Transform data and write to Apache Iceberg	https://docs.cloud.google.com/managed-spark/docs/tutorials/spark-to-iceberg
Overview	https://docs.cloud.google.com/managed-spark/docs/guides/spark-bigquery-connector
Query BigQuery tables	https://docs.cloud.google.com/managed-spark/docs/guides/query-bigquery-tables
Run PySpark code in BigQuery Studio notebooks	https://docs.cloud.google.com/managed-spark/docs/use-spark
Autoscale workload resources	https://docs.cloud.google.com/managed-spark/docs/concepts/autoscaling-serverless
Autotune Spark workloads	https://docs.cloud.google.com/managed-spark/docs/concepts/autotuning
Accelerate batch workloads and sessions with Lightning Engine	https://docs.cloud.google.com/managed-spark/docs/guides/lightning-engine-serverless
Run the Native Query Execution qualification tool	https://docs.cloud.google.com/managed-spark/docs/guides/run-nqe-tool
Serverless Spark solution accelerators	https://github.com/GoogleCloudPlatform/serverless-spark-workshop
Manage Spark dependencies	https://docs.cloud.google.com/managed-spark/docs/guides/manage-spark-dependencies
Customize Spark environment	https://docs.cloud.google.com/managed-spark/docs/guides/docker-yarn
Enable concurrent writes	https://docs.cloud.google.com/managed-spark/docs/guides/fileoutput-committer
Enhance Spark performance	https://docs.cloud.google.com/managed-spark/docs/guides/performance-enhancements
Tune Spark	https://docs.cloud.google.com/managed-spark/docs/support/spark-job-tuning
Use Lightning Engine	https://docs.cloud.google.com/managed-spark/docs/guides/lightning-engine
Use the console	https://docs.cloud.google.com/managed-spark/docs/quickstarts/create-cluster-console
Use the command line	https://docs.cloud.google.com/managed-spark/docs/quickstarts/create-cluster-gcloud
Create a cluster	https://docs.cloud.google.com/managed-spark/docs/quickstarts/create-cluster-template
Run a Spark job	https://docs.cloud.google.com/managed-spark/docs/quickstarts/submit-spark-job-template
Update a cluster	https://docs.cloud.google.com/managed-spark/docs/quickstarts/update-cluster-template
Delete a cluster	https://docs.cloud.google.com/managed-spark/docs/quickstarts/quickstart-explorer-delete
Use client libraries	https://docs.cloud.google.com/managed-spark/docs/quickstarts/create-cluster-client-libraries
Run Hadoop jobs	https://docs.cloud.google.com/managed-spark/docs/tutorials/bigquery-connector-mapreduce-example
Write and run Spark Scala jobs	https://docs.cloud.google.com/managed-spark/docs/tutorials/spark-scala
Run Trino	https://docs.cloud.google.com/managed-spark/docs/tutorials/trino-dataproc
Run Flink	https://docs.cloud.google.com/managed-spark/docs/concepts/components/flink
Run Hive	https://docs.cloud.google.com/managed-spark/docs/tutorials/kafka-tutorial
Run Pig	https://docs.cloud.google.com/managed-spark/docs/concepts/components/pig
Run HBase	https://docs.cloud.google.com/managed-spark/docs/tutorials/spark-hbase
Overview	https://docs.cloud.google.com/managed-spark/docs/tutorials/python-configuration
Migrate and set up Conda channels	https://docs.cloud.google.com/managed-spark/docs/guides/conda-channel-migration
Use Cloud Client Libraries for Python	https://docs.cloud.google.com/managed-spark/docs/tutorials/python-library-example
Overview	https://docs.cloud.google.com/managed-spark/docs/tutorials/bigquery-connector-spark-example
BigQuery connector code samples	https://docs.cloud.google.com/managed-spark/docs/examples/bigquery-example
Use the Cloud Storage connector	https://docs.cloud.google.com/managed-spark/docs/tutorials/gcs-connector-spark-tutorial
Use the Spark Spanner connector	https://docs.cloud.google.com/managed-spark/docs/tutorials/spanner-connector-spark-example
Explore and extract data	https://docs.cloud.google.com/managed-spark/docs/guides/explore-data
Transform data	https://docs.cloud.google.com/managed-spark/docs/guides/transform-data
Load data into BigQuery	https://docs.cloud.google.com/managed-spark/docs/guides/load-data
Create a lakehouse with Spark	https://docs.cloud.google.com/managed-spark/docs/guides/create-lakehouse
Query federated catalogs with Spark	https://docs.cloud.google.com/managed-spark/docs/guides/query-federated-catalogs
Configure metastores	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/secure-hive-metastore
Create an Apache Iceberg table with metadata in BigLake metastore	https://docs.cloud.google.com/managed-spark/docs/guides/iceberg-metadata-biglake-metastore
Use Iceberg	https://docs.cloud.google.com/managed-spark/docs/concepts/components/iceberg
Use Delta	https://docs.cloud.google.com/managed-spark/docs/concepts/components/delta
Use Hudi	https://docs.cloud.google.com/managed-spark/docs/concepts/components/hudi
Overview	https://docs.cloud.google.com/managed-spark/docs/concepts/notebooks
Run a Jupyter notebook on a cluster	https://docs.cloud.google.com/managed-spark/docs/tutorials/jupyter-notebook
Run a genomics analysis on a notebook	https://docs.cloud.google.com/managed-spark/docs/tutorials/genomics-single-cell
Use the JupyterLab extension to develop serverless Spark workloads	https://docs.cloud.google.com/managed-spark/docs/guides/jupyterlab-sessions-serverless
Use the Component Gateway	https://docs.cloud.google.com/managed-spark/docs/concepts/accessing/gateways
Connect to data sources	https://docs.cloud.google.com/managed-spark/docs/guides/connect-to-data-sources
Connect to Cloud Storage	https://docs.cloud.google.com/managed-spark/docs/concepts/connectors/cloud-storage
Connect to BigQuery	https://docs.cloud.google.com/managed-spark/docs/concepts/connectors/bigquery
Connect Hive to BigQuery	https://docs.cloud.google.com/managed-spark/docs/concepts/connectors/hive-bigquery
Connect to Bigtable	https://docs.cloud.google.com/managed-spark/docs/concepts/connectors/cloud-bigtable
Connect to Pub/Sub Lite	https://docs.cloud.google.com/managed-spark/docs/concepts/connectors/pubsub-lite
Fleet management	https://docs.cloud.google.com/managed-spark/docs/guides/fleet-management
Enable Spark data lineage	https://docs.cloud.google.com/managed-spark/docs/guides/spark-lineage
Enable Hive data lineage	https://docs.cloud.google.com/managed-spark/docs/guides/hive-lineage
Overview	https://docs.cloud.google.com/managed-spark/docs/concepts/components/overview
Delta Lake	https://docs.cloud.google.com/managed-spark/docs/concepts/components/delta
Docker	https://docs.cloud.google.com/managed-spark/docs/concepts/components/docker
Flink	https://docs.cloud.google.com/managed-spark/docs/concepts/components/flink
HBase	https://docs.cloud.google.com/managed-spark/docs/concepts/components/hbase
Hive WebHCat	https://docs.cloud.google.com/managed-spark/docs/concepts/components/hivewebhcat
Hudi	https://docs.cloud.google.com/managed-spark/docs/concepts/components/hudi
Iceberg	https://docs.cloud.google.com/managed-spark/docs/concepts/components/iceberg
Jupyter	https://docs.cloud.google.com/managed-spark/docs/concepts/components/jupyter
Pig	https://docs.cloud.google.com/managed-spark/docs/concepts/components/pig
Presto	https://docs.cloud.google.com/managed-spark/docs/concepts/components/presto
Install Ranger	https://docs.cloud.google.com/managed-spark/docs/concepts/components/ranger
Use Ranger with Kerberos	https://docs.cloud.google.com/managed-spark/docs/concepts/components/ranger-w-kerberos
Use Ranger with caching and downscoping	https://docs.cloud.google.com/managed-spark/docs/concepts/components/ranger-caching-downscoping
Back up and restore a Ranger schema	https://docs.cloud.google.com/managed-spark/docs/concepts/components/backup-ranger-schema
Solr	https://docs.cloud.google.com/managed-spark/docs/concepts/components/solr
Trino	https://docs.cloud.google.com/managed-spark/docs/concepts/components/trino
Zeppelin	https://docs.cloud.google.com/managed-spark/docs/concepts/components/zeppelin
Zookeeper	https://docs.cloud.google.com/managed-spark/docs/concepts/components/zookeeper
Overview	https://docs.cloud.google.com/managed-spark/docs/concepts/versioning/overview
Services	https://docs.cloud.google.com/managed-spark/docs/concepts/services
Overview	https://docs.cloud.google.com/managed-spark/docs/concepts/versioning/image-version-lists
3.0.x release image versions	https://docs.cloud.google.com/managed-spark/docs/concepts/versioning/image-release-3.0
2.3.x release image versions	https://docs.cloud.google.com/managed-spark/docs/concepts/versioning/image-release-2.3
2.2.x release image versions	https://docs.cloud.google.com/managed-spark/docs/concepts/versioning/image-release-2.2
2.1.x release image versions	https://docs.cloud.google.com/managed-spark/docs/concepts/versioning/image-release-2.1
Machine types	https://docs.cloud.google.com/managed-spark/docs/concepts/compute/supported-machine-types
GPUs	https://docs.cloud.google.com/managed-spark/docs/concepts/compute/gpus
Minimum CPU platform	https://docs.cloud.google.com/managed-spark/docs/concepts/compute/min-cpu
Flexible VMs	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/flexible-vms
Secondary workers	https://docs.cloud.google.com/managed-spark/docs/concepts/compute/secondary-vms
Local solid state drives	https://docs.cloud.google.com/managed-spark/docs/concepts/compute/local-ssds
Boot disks	https://docs.cloud.google.com/managed-spark/docs/concepts/compute/persistent-disks
Attached hyperdisks	https://docs.cloud.google.com/managed-spark/docs/concepts/compute/attached-disks
Overview	https://docs.cloud.google.com/managed-spark/docs/guides/create-cluster
Overview	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/network
Use secure tags	https://docs.cloud.google.com/managed-spark/docs/guides/use-secure-tags
Configure Private Service Connect	https://docs.cloud.google.com/managed-spark/docs/networking-private-service-connect
Select a cluster region	https://docs.cloud.google.com/managed-spark/docs/concepts/cluster-region
Enable auto zone placement	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/auto-zone
Create a multi-zone cluster	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/multi-zone
Schedule cluster deletion	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/scheduled-deletion
Use initialization actions	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/init-actions
Set cluster metadata	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/metadata
Set cluster properties	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/cluster-properties
Enable cluster web interfaces	https://docs.cloud.google.com/managed-spark/docs/concepts/accessing/cluster-web-interfaces
Create a high availability cluster	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/high-availability
Create a high obtainability cluster	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/high-obtainability
Create a node group cluster	https://docs.cloud.google.com/managed-spark/docs/guides/node-groups/driver-node-groups
Create a partial cluster	https://docs.cloud.google.com/managed-spark/docs/guides/create-partial-cluster
Create a zero-scale cluster	https://docs.cloud.google.com/managed-spark/docs/guides/create-zero-scale-cluster
Create a single-node cluster	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/single-node-clusters
Create sole-tenant cluster	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/sole-tenant-nodes
Create a custom image	https://docs.cloud.google.com/managed-spark/docs/guides/images
Set labels for filtering	https://docs.cloud.google.com/managed-spark/docs/guides/creating-managing-labels
Scale clusters manually	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/scaling-clusters
Autoscale clusters	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/autoscaling
Start and stop a cluster	https://docs.cloud.google.com/managed-spark/docs/guides/start-stop
Schedule cluster stop	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/scheduled-stop
Recreate a cluster	https://docs.cloud.google.com/managed-spark/docs/guides/recreate-cluster
Rotate clusters	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/cluster-rotation
Update and delete a cluster	https://docs.cloud.google.com/managed-spark/docs/guides/manage-cluster
Use SSH to connect to a cluster	https://docs.cloud.google.com/managed-spark/docs/concepts/accessing/ssh
Overview	https://docs.cloud.google.com/managed-spark/docs/guides/best-practices
Maximize cluster and job flexibility and efficiency	https://docs.cloud.google.com/managed-spark/docs/guides/maximize-cluster-job-flexibility
Hadoop data storage	https://docs.cloud.google.com/managed-spark/docs/concepts/hdfs
Select storage type	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/staging-bucket
Cache cluster data	https://docs.cloud.google.com/managed-spark/docs/concepts/cluster-caching
Offload shuffle data	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/enhanced-flexibility-mode
View cluster logs	https://docs.cloud.google.com/managed-spark/docs/guides/logging#cluster-logs
Life of a job	https://docs.cloud.google.com/managed-spark/docs/concepts/jobs/life-of-a-job
Restart jobs	https://docs.cloud.google.com/managed-spark/docs/concepts/jobs/restartable-jobs
Overview	https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/overview
Parameterization	https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/workflow-parameters
Use YAML files	https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/use-yamls
Use cluster selectors	https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/cluster-selectors
Use inline workflows	https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/inline-workflows
GitHub templates	https://docs.cloud.google.com/managed-spark/docs/guides/templates
Workflow scheduling solutions	https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/workflow-schedule-solutions
Use workflow templates	https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/use-workflows
Use Cloud Composer	https://docs.cloud.google.com/managed-spark/docs/tutorials/workflow-composer
Use Cloud Functions	https://docs.cloud.google.com/managed-spark/docs/tutorials/workflow-function
Use Cloud Scheduler	https://docs.cloud.google.com/managed-spark/docs/tutorials/workflow-scheduler
Create node pools	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/gke-nodepools
Recreate a GKE virtual cluster	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/gke-recreate-cluster
Create a GKE custom custom container image	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/gke-custom-images
Scale GKE clusters	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/gke-scale-cluster
Delete a GKE virtualcluster	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/gke-delete-cluster
Release versions	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/gke-versions
Visualize Spark DataFrames in notebooks	https://docs.cloud.google.com/managed-spark/docs/guides/visualize-dataframes-notebooks
Permissions and roles	https://docs.cloud.google.com/managed-spark/docs/concepts/iam-serverless
Personas and serverless IAM roles	https://docs.cloud.google.com/managed-spark/docs/concepts/personas-and-serverless-iam-roles
Use CMEK with Serverless for Apache Spark	https://docs.cloud.google.com/managed-spark/docs/guides/cmek-serverless
Default security measures	https://docs.cloud.google.com/managed-spark/docs/concepts/security-serverless
Use Secret Manager	https://docs.cloud.google.com/managed-spark/docs/guides/hadoop-google-secret-manager-credential-provider-serverless
Create custom constraints	https://docs.cloud.google.com/managed-spark/docs/guides/custom-constraints-serverless
Security best practices	https://docs.cloud.google.com/managed-spark/docs/concepts/security-best-practices
Authenticate to Managed Service for Apache Spark	https://docs.cloud.google.com/managed-spark/docs/authentication
Authenticate personal clusters	https://docs.cloud.google.com/managed-spark/docs/concepts/iam/personal-auth
Workforce identity federation	https://docs.cloud.google.com/managed-spark/docs/concepts/accessing/workforce-identity-federation
Permissions and roles	https://docs.cloud.google.com/managed-spark/docs/concepts/iam/iam
Managed Service for Apache Spark principals	https://docs.cloud.google.com/managed-spark/docs/concepts/iam/principals
Granular IAM	https://docs.cloud.google.com/managed-spark/docs/concepts/iam/granular-iam
Assign roles for Kubernetes	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/gke-iam
Service accounts	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/service-accounts
Secure multi-tenancy using Kerberos	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/security
Secure multi-tenancy using service accounts	https://docs.cloud.google.com/managed-spark/docs/concepts/iam/sa-multi-tenancy
Encrypt memory	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/confidential-compute
Manage data encryption keys	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/customer-managed-encryption
Enable Ranger authorization service	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/ranger-plugin
Use the Secret Manager credential provider	https://docs.cloud.google.com/managed-spark/docs/guides/hadoop-google-secret-manager-credential-provider
Create and secure a Hive metastore cluster	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/secure-hive-metastore
Create custom constraints	https://docs.cloud.google.com/managed-spark/docs/concepts/custom-constraints
Assured Workloads	https://docs.cloud.google.com/assured-workloads/docs
FedRAMP Compliance	https://docs.cloud.google.com/managed-spark/docs/concepts/fedramp-compliance
Use Kerberos	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/security
VPC Service Controls	https://docs.cloud.google.com/managed-spark/docs/concepts/configuring-clusters/network#dataproc-vpc-sc-networks
Set Workforce access	https://docs.cloud.google.com/managed-spark/docs/concepts/accessing/workforce-identity-federation
Investigate workloads with AI-powered Gemini Cloud Assist	https://docs.cloud.google.com/managed-spark/docs/guides/ai-investigations-with-gca
Monitor batch workloads	https://docs.cloud.google.com/managed-spark/docs/guides/monitor-troubleshoot-batches
Monitor interactive sessions	https://docs.cloud.google.com/managed-spark/docs/guides/monitor-troubleshoot-sessions
Spark metrics	https://docs.cloud.google.com/managed-spark/docs/concepts/metrics-serverless
Enable data lineage	https://docs.cloud.google.com/managed-spark/docs/guides/data-lineage
Profile resource usage	https://docs.cloud.google.com/managed-spark/docs/guides/profiling
Resource Manager audit logs	https://docs.cloud.google.com/managed-spark/docs/guides/dataprocrm-audit-logging
Troubleshoot batch and session creation failures	https://docs.cloud.google.com/managed-spark/docs/support/batch-session-creation-failures
Troubleshoot batch and session connectivity	https://docs.cloud.google.com/managed-spark/docs/support/troubleshoot-connectivity
Investigate clusters and jobs with AI-powered Gemini Cloud Assist	https://docs.cloud.google.com/managed-spark/docs/guides/gca-clusters
Overview	https://docs.cloud.google.com/managed-spark/docs/guides/monitor-clusters-jobs
Managed Service for Apache Spark metrics	https://docs.cloud.google.com/managed-spark/docs/guides/metrics
Create metric alerts	https://docs.cloud.google.com/managed-spark/docs/guides/alerts
Profile resource usage	https://docs.cloud.google.com/managed-spark/docs/guides/profiling
Managed Service for Apache Spark logs	https://docs.cloud.google.com/managed-spark/docs/guides/logging
Job output logs	https://docs.cloud.google.com/managed-spark/docs/guides/job-output
Audit logs	https://docs.cloud.google.com/managed-spark/docs/guides/audit-logging
View cluster diagnostic data	https://docs.cloud.google.com/managed-spark/docs/support/diagnose-clusters
Troubleshoot cluster creation issues	https://docs.cloud.google.com/managed-spark/docs/support/troubleshoot-cluster-creation
Troubleshoot jobs	https://docs.cloud.google.com/managed-spark/docs/concepts/jobs/troubleshoot-jobs
Troubleshoot memory errors	https://docs.cloud.google.com/managed-spark/docs/support/troubleshoot-oom-errors
Troubleshoot job delays	https://docs.cloud.google.com/managed-spark/docs/concepts/jobs/troubleshoot-job-delays
View job history	https://docs.cloud.google.com/managed-spark/docs/concepts/jobs/history-server
Troubleshoot workflow templates	https://docs.cloud.google.com/managed-spark/docs/concepts/workflows/debugging
Enable Kubernetes logging	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/gke-logging
Diagnose Kubernetes clusters	https://docs.cloud.google.com/managed-spark/docs/guides/dpgke/gke-diagnose-cluster
Managed Service for Apache Spark MCP server	https://docs.cloud.google.com/managed-spark/docs/guides/use-dataproc-mcp
Code samples	https://docs.cloud.google.com/dataproc/docs/samples
Get Started with Google Cloud	https://docs.cloud.google.com/docs/get-started
Product List	https://docs.cloud.google.com/docs/product-list
Cloud Customer Care	https://docs.cloud.google.com/support/docs
Agent Platform	https://docs.cloud.google.com/products/gemini-enterprise-agent-platform
Apigee API Management	https://docs.cloud.google.com/apigee
BigQuery	https://docs.cloud.google.com/bigquery
Compute Engine	https://docs.cloud.google.com/products/compute
Cloud CDN	https://docs.cloud.google.com/cdn
Cloud Run	https://docs.cloud.google.com/run
Cloud Storage	https://docs.cloud.google.com/storage
Cloud SQL	https://docs.cloud.google.com/sql
Gemini Enterprise	https://docs.cloud.google.com/gemini/enterprise/docs
Google Kubernetes Engine	https://docs.cloud.google.com/kubernetes-engine
Looker	https://docs.cloud.google.com/looker
Access and resources management	https://docs.cloud.google.com/docs/access-resources
Costs and usage management	https://docs.cloud.google.com/docs/costs-usage
Infrastructure as code	https://docs.cloud.google.com/docs/iac
SDK, languages, frameworks, and tools	https://docs.cloud.google.com/docs/devtools
AI and ML	https://docs.cloud.google.com/docs/ai-ml
Application development	https://docs.cloud.google.com/docs/application-development
Application hosting	https://docs.cloud.google.com/docs/application-hosting
Compute	https://docs.cloud.google.com/docs/compute-area
Data analytics and pipelines	https://docs.cloud.google.com/docs/data
Databases	https://docs.cloud.google.com/docs/databases
Distributed, hybrid, and multicloud	https://docs.cloud.google.com/docs/dhm-cloud
Industry solutions	https://docs.cloud.google.com/docs/industry
Migration	https://docs.cloud.google.com/docs/migration
Networking	https://docs.cloud.google.com/docs/networking
Observability and monitoring	https://docs.cloud.google.com/docs/observability
Security	https://docs.cloud.google.com/docs/security
Storage	https://docs.cloud.google.com/docs/storage
Home	https://docs.cloud.google.com/
Documentation	https://docs.cloud.google.com/docs
Data analytics	https://docs.cloud.google.com/docs/data
Managed Service for Apache Spark	https://docs.cloud.google.com/managed-spark/docs
Guides	https://docs.cloud.google.com/managed-spark/docs/overview/key-concepts
Cloud Shell walkthrough	https://docs.cloud.google.com/shell/docs/tutorials
Google Cloud client libraries for Python	https://docs.cloud.google.com/python/docs/reference/dataproc/latest
Managed Service for Apache Spark gRPC APIs	https://docs.cloud.google.com/managed-spark/docs/reference/rpc
GoogleCloudPlatform/python-dataproc	https://github.com/googleapis/google-cloud-python/tree/main/packages/google-cloud-dataproc/samples/generated_samples
inline workflow	https://docs.cloud.google.com/managed-spark/docs/reference/rpc/google.cloud.dataproc.v1#google.cloud.dataproc.v1.WorkflowTemplateService.InstantiateInlineWorkflowTemplate
instantiate_inline_workflow_template.py	https://github.com/googleapis/google-cloud-python/blob/main/packages/google-cloud-dataproc/samples/generated_samples/dataproc_v1_generated_workflow_template_service_instantiate_inline_workflow_template_async.py
Open in Cloud Shell	https://ssh.cloud.google.com/cloudshell/open?cloudshell_git_repo=https://github.com/googleapis/python-dataproc&cloudshell_working_dir=samples/snippets&tutorial=python-api-walkthrough.md&cloudshell_open_in_editor=submit_job_to_cluster.py
Cloud Shell walkthrough	https://docs.cloud.google.com/managed-spark/docs/tutorials/python-library-example#run-walkthrough
service account credentials	https://docs.cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually
CreateCluster	https://docs.cloud.google.com/managed-spark/docs/reference/rpc/google.cloud.dataproc.v1#google.cloud.dataproc.v1.ClusterController.CreateCluster
SubmitJob	https://docs.cloud.google.com/managed-spark/docs/reference/rpc/google.cloud.dataproc.v1#google.cloud.dataproc.v1.JobController.SubmitJob
DeleteCluster	https://docs.cloud.google.com/managed-spark/docs/reference/rpc/google.cloud.dataproc.v1#google.cloud.dataproc.v1.JobController.DeleteCluster
Creative Commons Attribution 4.0 License	https://creativecommons.org/licenses/by/4.0/
Apache 2.0 License	https://www.apache.org/licenses/LICENSE-2.0
Google Developers Site Policies	https://developers.google.com/site-policies
See all products	https://cloud.google.com/products/
Google Cloud pricing	https://cloud.google.com/pricing/
Google Cloud Marketplace	https://cloud.google.com/marketplace/
Contact sales	https://cloud.google.com/contact/
Community forums	https://discuss.google.dev/c/google-cloud/14/
Support	https://cloud.google.com/support-hub/
Release Notes	https://docs.cloud.google.com/release-notes
System status	https://status.cloud.google.com
GitHub	https://github.com/googlecloudPlatform/
Getting Started with Google Cloud	https://docs.cloud.google.com/docs/get-started/
Code samples	https://docs.cloud.google.com/docs/samples
Cloud Architecture Center	https://docs.cloud.google.com/architecture/
Training and Certification	https://cloud.google.com/learn/training/
Builders Hub	https://me.developers.google.com/
Blog	https://cloud.google.com/blog/
Events	https://cloud.google.com/events/
X (Twitter)	https://x.com/googlecloud
Google Cloud on YouTube	https://www.youtube.com/googlecloud
Google Cloud Tech on YouTube	https://www.youtube.com/googlecloudplatform
	https://developers.google.com/
Android	https://developer.android.com/
Chrome	https://developer.chrome.com/
Firebase	https://firebase.google.com/
Google AI	https://ai.google.com/
About Google	https://about.google/
Privacy	https://policies.google.com/privacy
Site terms	https://policies.google.com/terms?hl=en
Google Cloud terms	https://cloud.google.com/product-terms
Manage cookies	https://docs.cloud.google.com/managed-spark/docs/tutorials/python-library-example
Our third decade of climate action: join us	https://cloud.google.com/sustainability
Subscribe	https://cloud.google.com/newsletter/

Viewport: width=device-width, initial-scale=1

URLs of crawlers that visited me.