Processing 2T tokens on CPUs took days. GPU acceleration cut prep to hours, unlocking a 5% accuracy gain.
A Korean AI startup developing sovereign large language models required high-quality training datasets for a language with scarce public resources.
Processing over two trillion tokens on CPUs created severe bottlenecks, with data curation tasks like deduplication taking days to complete. This...
“Deduplication is one of the most time-consuming processes when handling a very large dataset. The time saved from NeMo Curator GPU acceleration was the most significant benefit we experienced from using the library.”
Proprietary large language models for Korean and Asian languages.
NVIDIA is a technology company that specializes in semiconductors, graphics processing units, and artificial intelligence for applications in data centers, gaming, and more.
Related implementations across industries and use cases
Captioning 10,000 images consumed 333 hours of human effort. AI now completes the workflow in under 15 hours.
Japanese layouts and honorifics baffled the system. Calibrating a local LLM raised form accuracy 12% and dialogue success to 87%.
Engineers spent 90% of time on data prep. New pipelines flipped that to 90% modeling and cut tuning from 7 days to 1 hour.
Engineers manually correlated alerts across systems. AI agents now diagnose issues and suggest fixes, cutting recovery time by 35%.
Minor edits required days of crew coordination. Now, staff use avatars to modify dialogue and translate languages instantly.
Lab supply orders were handwritten in notebooks. Digital ordering now takes seconds, saving 30,000 hours for research annually.
Experts spent 15 minutes pulling data from scattered systems. Natural language prompts now generate detailed reports instantly.