NCA Generative AI LLM (NCA-GENL) Practice Exam

The metric that indicates contextual coherence and semantic richness within a model is BERTScore. This metric utilizes contextual embeddings from the BERT model to evaluate the semantic similarity between the generated text and reference text. BERTScore captures nuances in language by considering the embedding of words in their specific contexts, which allows it to better assess the depth and coherence of generated outputs compared to more traditional metrics.

While BLEU and ROUGE scores are commonly used for evaluating generative models, they primarily focus on n-gram overlaps and do not capture the contextual relationships among words or phrases as effectively as BERTScore does. Accuracy Rate, on the other hand, typically measures correctness against ground truth labels and does not pertain to the quality of generated text in terms of coherence or richness. Thus, BERTScore stands out as the metric that best measures the semantic and contextual quality of generated text.