AWS Certified Machine Learning – Specialty Set 7 Welcome to AWS Certified Machine Learning - Specialty Set 7. Please enter your email details to get QUIZ Details on your email id. Click on Next Button to proceed. 1. In a regression problem, if we plot the residuals in a histogram and observe a distribution heavily skewed to the right of zero indicating mostly positive residuals, what does this mean? Our model is sufficient with regard to aggregate residual. Our model is consistent underestimating. Our model is consistently overestimating.2. We want to perform automatic model tuning on our linear learner model. We have chosen the tunable hyperparameter we want to use. What is our next step? Decide what hyperparameter we want SageMaker to tune in the tuning process. Choose a range of values which SageMaker will sweep through during the tuning process. Choose a target objective metric we want SageMaker to use in the tuning process.3. In your first training job of a regression problem, you observe an RMSE of 3.4. You make some adjustments and run the training job again, which results in an RMSE of 2.2. What can you conclude from this? The adjustments made your model recall worse. The adjustments improved your model accuracy. The adjustments had no effect on your model accuracy.4. You are designing a testing plan for an update release of your company's mission critical loan approval model. Due to regulatory compliance, it is critical that the updates are not used in production until regression testing has shown that the updates perform as good as the existing model. Which validation strategy would you choose? (Choose 2) Make use of backtesting with historic data. Use an A/B test to expose the updates to real-world traffic. Use a rolling upgrade to determine if the model is ready for production. Use a K-Fold validation method.5. A colleague is preparing for their very first training job using the XGBoost algorithm. They ask you how they can ensure that training metrics are captured during the training job. How do you direct them? Do nothing. Use SageMaker's built-in logging feature and view the logs using Quicksight. Do nothing. Sagemaker's built-in algorithms are already configured to send training metrics to CloudWatch. Do nothing. Sagemaker's built-in algorithms are already configured to send training metrics to CloudTrail.6. Which of the following metrics are recommended for tuning a Linear Learner model so that we can help avoid overfitting? (Choose 3) test:precision validation:precision validation:recall validation:objective_loss test:recall7. We have just completed a validation job for a multi-class classification model that attempts to classify books into one of five genres. In reviewing the validation metrics, we observe a Macro Average F1 score of 0.28 with one genre, historic fiction, having an F1 score of 0.9. What can we conclude from this? We might try a linear regression model instead of a multi-class classification. Our model is very poor at predicting historic fiction but quite good at the other genres given the Macro F1 Score. Our training data might be biased toward historic fiction and lacking in examples of other genres.8. After training and validation sessions, we notice that the error rate is higher than we want for both sessions. Visualization of the data indicates that we don't seem to have any outliers. What else might we do? (Choose 3) Add more variables to the dataset. Run training for a longer period of time. Run a random cut forest algorithm on the data. Gather more data for our training process. Reduce the dimensions of the data.9. After training and validation sessions, we notice that the accuracy rate for training is acceptable but the accuracy rate for validation is very poor. What might we do? (Choose 3) Encode the data using Laminar Flow Step-up. Reduce dimensionality. Add an early stop. Increase the learning rate. Gather more data for our training process.10. In your first training job of a binary classification problem, you observe an F1 score of 0.996. You make some adjustments and rerun the training job again, which results in an F1 score of 0.034. What can you conclude from this? (Choose 2) The adjustments drastically improved our model. The adjustments drastically worsened our model. Nothing can be concluded from an F1 score by itself. Our accuracy has decreased.11. After multiple training runs, you notice that the the loss function settles on different but similar values. You believe that there is potential to improve the model through adjusting hyperparameters. What might you try next? Decrease the learning rate. Change from a CPU instances to a GPU instance. Change to another algorithm.12. In a binary classification problem, you observe that precision is poor. Which of the following most contribute to poor precision? Type III Error Type IV Error Type I Error13. You are preparing for a first training run using a custom algorithm that you have prepared in a docker container. What should you do to ensure that the training metrics are visible to CloudWatch? Enable Kinesis Streams to capture the log stream emitting from the custom algorithm containers. When defining the training job, ensure that the metric_definitions section is populated with relevant metrics from the stdout and stderr streams in the container. Enable CloudTrail for the respective container to capture the relevant training metrics from the custom algorithm.14 out of Please fill in the comment box below.