Colab Notebook: Evaluate Gemma 2 with the Gen AI evaluation service on Vertex AI → https://goo.gle/4iIAZpI
Docs: Gen AI evaluation service on Vertex AI → https://goo.gle/3VOThvJ
Reference: Model-based metrics prompts → https://goo.gle/3P4s7ND
Learn how to evaluate open models with the Gen AI evaluation service on Vertex AI. Follow along as Googlers Wietse Venema and Ivan Nardini present a Colab notebook that shows how to evaluate Gemma 2 (an open model) using the XSum dataset. It covers computation-based metrics (ROUGE, F1-score) and model-based metrics (using Gemini as a judge). The use cases discussed in this video include model selection, fine-tuning evaluation, generation optimization, prompt engineering, and prompt migration.
Watch more Google Cloud: Building with Hugging Face → https://goo.gle/BuildWithHuggingFace
Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
#GoogleCloud #HuggingFace
Speakers: Wietse Venema, Ivan Nardini
Products Mentioned: Vertex AI, Gemini
