Google Certified Professional Data Engineer Set 1 Author: CloudVikas Published Date: 18 June 2021 1 Comment on Google Certified Professional Data Engineer Set 1 Welcome to Google Certified Professional Data Engineer Set 1.Click on Next Button to proceed. 1. You want to use Pub/Sub to distribute jobs to a group of Compute Engine VMs, which should each take the next job from the queue. How should you configure Pub/Sub?Create a new Topic for each individual job. Create multiple Subscriptions for each Topic, one for every Compute Engine VM in the group.Create a new Topic for each individual job. Create a Subscription for each Topic that can be shared by every Compute Engine VM in the group.Create a Topic for jobs. Create a Subscription for this Topic that can be shared by every Compute Engine VM in the group.2. A customer wants to run Spark jobs on a low-cost ephemeral Dataproc cluster, utilizing preemptible workers wherever possible, but needs to store the results of Dataproc jobs persistently. What would you recommend?Do not use preemptible workers at all, it will prevent you from choosing any persistent storage option.Use a secondary group of preemptible worker nodes, but add custom code to a job that copies its results to Cloud Storage.Use the Cloud Storage connector, and specify GCS locations for the input and ouput of jobs.3. What is a federated data source?A BigQuery data set that belongs to a different GCP project, used in an SQL statement.An external data source that can be queried directly even though the data is not stored in BigQuery.A BigQuery table that belongs to a different data set, used in an SQL statement.4. Which transformation can be used to process collections of key/value pairs, in a similar fashion to the shuffle phase of a map/shuffle/reduce-style algorithm?GroupByKeyPartitionCoGroupByKey5. Cloud Memorystore is essentially a managed service based on which open-source project?MongoDBRedisMemcached6. Which big data programming model is implemented with Cloud Dataflow?Apache NiFiApache BeamApache Spark7. Which GCP product implements the Apache Beam SDK and is sometimes recommended as an alternative to Dataproc particularly for streaming data?Cloud ComposerCloud DataflowCloud Datalab8. Which native output connectors are supported by Dataproc?Which native output connectors are supported by Dataproc? (Choose 3) BigQuery Cloud Bigtable Cloud Storage Cloud Firestore Cloud SQL9. You are required to share a subset of data from a BigQuery data set with a 3rd party analytics team. The data may change over time, and you should not grant unnecessary projects permissions to this team if you can avoid it. How should you proceed?Create an authorized view based on a specific query for the subset of data, and provide access for the team only to that view.Create an export of the data subset to a Cloud Storage bucket. Provide a signed URL for the team to download the data from the bucket.Add the team to your GCP project and assign them the BigQuery Data Viewer IAM role for the data set.10. What is the purpose of a trigger in Cloud Dataflow?Triggers determine when to initiate the processing of a pipeline.Triggers determine when to emit output data, and behave differently for bounded and unbounded data.Triggers determine when to emit output data, but only apply to unbounded/streaming input data.11. What is the minimum retention duration for a message in Pub/Sub?1 minute10 minutes1 day12. What could you consider to improve BigQuery performance for queries that use filters or aggregation against particular columns? Use clustering on non-partitioned tables. Use clustered partitioned tables Use partitioned tables13. What is the compute model of Cloud Dataflow?Dataflow fully manages Compute Engine services to run jobs.Dataflow manages Cloud Storage but requires a pre-built Kubernetes Engine cluster to run jobs.Dataflow requires a pre-built Cloud Dataproc cluster to run jobs.14. Using Cloud IAM, what is the most granular level for which you can configure access control for Pub/Sub?Across individual topcis and subscriptionsAcross individual topics, but not subscriptionsAcross the entire project, including all topics and subscriptions15. Pub/Sub is designed for what main purpose?To decouple services and distribute workloadsTo enable asynchronous workflowsTo receive, buffer and distribute eventsAll of the above16. What is the name given to a dataset that can be acted upon within a Cloud Dataflow pipeline?BucketPCollectionAggregation17. Cloud Storage bucket names must be unique within:All GCP projectsA regionA projectA zone18. High Availability for Cloud SQL PostgreSQL instances works because:Instances share a regional replicated persistent diskInstances replicate data directly between themselves for automatic failoverGoogle SREs are on call and can quickly bring up a new instance in the event of a failure19. Which operation can be used to invoke a user-specified function of each element of an input PCollection?User-defined functions (UDFs)GroupByKeyParDo20. Your customer would like to use Dataproc, but the standard image does not contain some additional Spark components required to run their jobs. What would you recommend?Use a Dataproc cluster, but specify an initialization action that installs all of the additional components.Create custom Dataproc image that fulfils the customer requirements and use it to deploy a Dataproc cluster.Create an image that fulfils the customer requirements and use it to deploy a custom Spark cluster using Compute Engine.21. If you have 2 replicating clusters in your Bigtable instance, how can you ensure that your application will be guaranteed strong consistency for its transactions?Use an application profile that specifies single-cluster routing.Refactor your application so that strong consistency is no longer required.Use an application profile that specifies multi-cluster routing, but place both clusters within the same region.22. True of False: Preemptible workers in a Dataproc cluster cannot store HDFS data.TrueFalse23. What action should you take if you observe that Bigtable performance is poor on a subset of queries?Use the Key Visualizer tool to identify hot spots and consider changing how the row key distributes rowsAdd debugging steps to your application to identify the problematic queriesAdd an additional cluster to the instance to increase the read and write throughput24. You are required to load a large volume of data into BigQuery that does contain some duplication. What should you do to ensure the best query performance once the data is loaded?Denormalize the data by using nested or repeated fields.Normalize the data to eliminate duplicates from being stored, and reformat the data as JSON.Normalize the data to eliminate duplicates from being stored, and reformat the data as Avro.25. How does Bigtable managed rows, tables and tablets?Tables are sorted into contiguous rows, and stored in a single tabletTables are sorted into contiguous rows, and sharded into tabletsTables contain unsorted rows, and stored in a single tablet26. How does Bigtable managed the storage of tablets?Tablets are stored on cluster nodes, but storage can dynamically grow as part of the managed serviceTablets are stored on cluster nodes, which must be sized accordingly for their storage needsTablets are stored in Google Colossus, but a cluster node has a limit on how much storage it can process27. What method could you use to help compute averages when dealing with unbounded/streaming data?Change the input data to be batched/bounded to make it easier to compute averagesSession windowsSliding windows28. What is the best way to grant someone temporary access to a file if they do not have a Google account?Use a signed URL with a specific expiryMake the file publicly accessible but ask the user not to share the linkUse a long and complex name for the bucket and object29. When you run a query in BigQuery, what happens to the results?Query results exist only within the BigQuery UI and are not stored in a table.Query results are either written to a destination table specified by the user, or to a temporary cached results table.Query results exist only in BigQuery memory and are not stored in a table.30. Which features are not compatible with Dataproc autoscaling?Which features are not compatible with Dataproc autoscaling? (Choose 3) Spark Structured Streaming Preemptible workers High-availability clusters MapReduce tasks HDFS Storage31 out of 30Please fill in the comment box below. Email Author: CloudVikas
