Groups Similar Look up By Text Browse About



Similar articles
Article Id Title Prob Score Similar Compare
218014 ZDNET 2021-10-11:
Microsoft and Nvidia create 105-layer, 530 billion parameter language model that needs 280 A100 GPUs, but it's still biased
1.000 Find similar Compare side-by-side
217934 VENTUREBEAT 2021-10-11:
Microsoft and Nvidia team up to train one of the world’s largest language models
0.952 0.665 Find similar Compare side-by-side
217942 VENTUREBEAT 2021-10-11:
Microsoft taps AI techniques to bring Translator to 100 languages
0.050 0.568 Find similar Compare side-by-side
217885 ZDNET 2021-10-13:
Microsoft Translator now works across 103 languages
0.003 0.422 Find similar Compare side-by-side
218175 ZDNET 2021-10-14:
The state of AI in 2021: Machine learning in production, MLOps and data-centric AI
0.315 Find similar Compare side-by-side
217957 VENTUREBEAT 2021-10-11:
Facebook quietly acquires synthetic data startup AI.Reverie
0.283 Find similar Compare side-by-side
217772 ZDNET 2021-10-13:
Global Azure outage knocked out virtual machines, other VM-dependent services
0.257 Find similar Compare side-by-side
217670 ZDNET 2021-10-8:
EU regulators said to be asking Teams rivals about Microsoft's bundling practices, per Slack's 2020 complaint
0.243 Find similar Compare side-by-side
218122 VENTUREBEAT 2021-10-14:
Facebook introduces dataset and benchmarks to make AI more ‘egocentric’
0.239 Find similar Compare side-by-side
217928 TECHREPUBLIC 2021-10-11:
Python ends C and Java's 20-year reign atop the TIOBE index
0.238 Find similar Compare side-by-side
218078 ARSTECHNICA 2021-10-14:
How a mass extinction resulted in the rise of the snakes
0.236 Find similar Compare side-by-side
218011 ZDNET 2021-10-11:
Amazon AWS's AI team seeks the profound in the industrial
0.234 Find similar Compare side-by-side
217770 VENTUREBEAT 2021-10-12:
Cloud optimization startup Cast AI raises $10M
0.233 Find similar Compare side-by-side
217839 THEVERGE 2021-10-13:
Acer’s Chromebook 515 is made for business on the go
0.232 Find similar Compare side-by-side
217875 VENTUREBEAT 2021-10-12:
DeepMind is developing one algorithm to rule them all
0.228 Find similar Compare side-by-side
217925 VENTUREBEAT 2021-10-10:
AI lab DeepMind becomes profitable and bolsters relationship with Google
0.221 Find similar Compare side-by-side
218023 ZDNET 2021-10-12:
Microsoft warns over password attacks against these Office 365 customers
0.214 Find similar Compare side-by-side
217948 VENTUREBEAT 2021-10-11:
DeepMind proposes new benchmark to improve robots’ object-stacking abilities
0.214 Find similar Compare side-by-side
217834 ARSTECHNICA 2021-10-8:
Shareholders pressure Microsoft into expanding its right-to-repair efforts
0.213 Find similar Compare side-by-side
217776 VENTUREBEAT 2021-10-12:
AI edge chip startup Hailo lands $136M
0.211 Find similar Compare side-by-side
217821 TECHREPUBLIC 2021-10-13:
How to secure Microsoft 365 with app governance
0.211 Find similar Compare side-by-side
218024 ZDNET 2021-10-12:
Jabra announces new Evolve2 75 with advanced ANC and focus on hybrid work
0.210 Find similar Compare side-by-side
218036 ZDNET 2021-10-12:
Microsoft delivers near-final VS 2022 Release Candidate and designates November 8 as GA date
0.207 Find similar Compare side-by-side
217836 ZDNET 2021-10-13:
Microsoft announces expansion of Emissions Impact Dashboard
0.207 Find similar Compare side-by-side
217579 THEVERGE 2021-10-8:
The Nintendo Switch OLED model is now available online
0.205 Find similar Compare side-by-side

1

ID: 218014

URL: https://www.zdnet.com/article/microsoft-and-nvidia-create-105-layer-530-billion-parameter-language-model-that-needs-280-a100-gpus-and-its-biased/

Date: 2021-10-11

Microsoft and Nvidia create 105-layer, 530 billion parameter language model that needs 280 A100 GPUs, but it's still biased

Tech giants have come up with the 'most powerful monolithic transformer language model trained to date', but it still suffers from bias. Nvidia and Microsoft have teamed up to create the Megatron-Turing Natural Language Generation model, which the duo claims is the "most powerful monolithic transformer language model trained to date". The AI model has 105 layers, 530 billion parameters, and operates on chunky supercomputer hardware like Selene. By comparison, the vaunted GPT-3 has 175 billion parameters. " Each model replica spans 280 NVIDIA A100 GPUs, with 8-way tensor-slicing within a node, and 35-way pipeline parallelism across nodes," the pair said in a blog post. The model was trained on 15 datasets that contained 339 billion tokens, and was capable of showing how larger models need less training to operate well. However, the need to operate with languages and samples from the real world meant an old problem with AI reappeared: Bias. " While giant language models are advancing the state of the art on language generation, they also suffer from issues such as bias and toxicity," the duo said. "Our observations with MT-NLG are that the model picks up stereotypes and biases from the data on which it is trained. Microsoft and Nvidia are committed to working on addressing this problem. "Our observations with MT-NLG are that the model picks up stereotypes and biases from the data on which it is trained. Microsoft and Nvidia are committed to working on addressing this problem. " It wasn't so long ago that Microsoft had its chatbot Tay turn full Nazi in a matter of hours by interacting on the internet.