Tag

#model-evaluation

Every item tagged model-evaluation, newest first.

2 items

OpenAI and Anthropic share findings from a joint safety evaluation

OpenAI and Anthropic conducted a joint safety evaluation of each other's models, testing for misalignment, instruction following, and other safety risks. The evaluation highlighted both progress and challenges in AI safety. You can learn from their findings to improve your own model's safety. The collaboration demonstrates the value of cross-lab testing in advancing AI safety.

Key takeaways

Joint evaluation tested models for misalignment and safety risks.
Findings highlighted both progress and challenges in AI safety.
Cross-lab collaboration seen as valuable for advancing safety.

OOpenAI#ai-safety #model-evaluation #collaboration

researchJul 7

Evaluating large language models trained on code

OpenAI published a study evaluating large language models trained on code, comparing performance across tasks like code completion, bug detection, and code summarization. The study assesses model performance on a range of programming languages and finds that larger models generally perform better but with diminishing returns. You can use these results to inform your model selection for code-related tasks.

Key takeaways

Larger models show better performance on code tasks but with diminishing returns.
Evaluated across multiple programming languages.
Useful for builders selecting models for code-related tasks.

OOpenAI#code-llm #model-evaluation #programming-languages