OpenAI indicates that GPT-6 has enhanced its capabilities in legacy programming languages such as COBOL, Fortran, and Visual Basic, and has made progress in two evaluations. The disclosed results include: GPT-6 Astra achieved an average test pass rate of 87.4% across 20 programming languages, and GPT-6.1 Sol had an accuracy rate of 97.9% on the COBOL migration benchmark.

These languages have fewer available training codes than languages like Python, but OpenAI indicates that they support critical global infrastructure. IBM estimates that these languages support over 40% of online banking systems and process 95% of ATM transactions.

OpenAI developed two evaluations to measure the progress of the model: one tests the ability to build databases across programming languages, and the other assesses the capability to translate COBOL into other languages.

OpenAI How to Test the Legacy Programming Skills of GPT-6

MultiLangBench requires the model to create a database engine compatible with SQLite from scratch. OpenAI reports that when using the highest level of inference intensity, GPT-6's average pass rate across 20 languages reaches 87.4%.

In contrast, the performance of GPT-5.6 and Terra was 37.9%. GPT-6 and Luna recorded 40.3% in the same evaluation, which is slightly higher than that of Terra. The comparison results provided by OpenAI show that the improvement rates of different models are not the same.

Another indicator that measures the programming performance left by GPT-6 focuses on code migration. The COBOL benchmark tests the translation of code into C++, Java, Kotlin, Python, or Rust, rather than building a database engine.

In this evaluation, GPT-6.1 and Sol achieved an accuracy of 86.6% at a lower level of reasoning intensity, while GPT-5.6 and Sol only reached 5.4%. After increasing the reasoning intensity, the accuracy of GPT-6.1 and Sol rose to 97.9%.

This article was generated with the assistance of artificial intelligence and has been reviewed by an editorial team.