[ML News] DeepMind builds Gopher | Google builds GLaM | Suicide capsule uses AI to check access

#mlnews #gopher #glam

Your updates on everything going on in the Machine Learning world.

Sponsor: Weights & Biases
https://wandb.me/yannic

OUTLINE:
0:00 - Intro & Overview
0:20 - Sponsor: Weights & Biases
3:05 - DeepMind releases 3 papers on large language models
11:45 - Hugging Face Blog: Training CodeParrot from scratch
14:25 - Paper: Pre-Training vision systems with noise
15:45 - DeepMind advances Quantum Mechanics
16:45 - GoogleAI trains GLaM: 1 Trillion Parameters Mixture of Experts Model
18:45 - Colin Raffel calls for building ML models like we build Open-Source software
22:05 - A rebuke of the hype around DeepMind's math paper
24:45 - Helpful Things
32:25 - Suicide Capsule plans AI to assess your mental state before use
35:15 - Synthesia raises 50M to develop AI avatars


Weights & Biases Embedding Projector
https://twitter.com/_ScottCondron/status/1469411468139536385?utm_source=pocket_mylist
https://docs.wandb.ai/ref/app/features/panels/weave/embedding-projector
https://wandb.ai/timssweeney/toy_datasets/reports/Feature-Report-W-B-Embeddings-Projector--VmlldzoxMjg2MjY4?accessToken=bo36zrgl0gref1th5nj59nrft9rc4r71s53zr2qvqlz68jwn8d8yyjdz73cqfyhq

DeepMind releases 3 papers on large language models
https://deepmind.com/blog/article/language-modelling-at-scale
https://arxiv.org/pdf/2112.04426.pdf
https://kstatic.googleusercontent.com/files/b068c6c0e64d6f933068f7de30ea722359ef87c6c14d3065856b86d44fbdf2dea3ff373ed9eb751514f242d20df9d6a468622fad093f962563545e7d0cdb9dba
https://arxiv.org/pdf/2112.04359.pdf
https://deepmind.com/research/publications/2021/improving-language-models-by-retrieving-from-trillions-of-tokens

Hugging Face Blog: Training CodeParrot from scratch
https://huggingface.co/blog/codeparrot?utm_source=pocket_mylist

Paper: Pre-Training vision systems with noise
https://mbaradad.github.io/learning_with_noise/

DeepMind advances Quantum Mechanics
https://deepmind.com/blog/article/Simulating-matter-on-the-quantum-scale-with-AI
https://storage.googleapis.com/deepmind-media/papers/Data_Driven_Density_Functional_Design/data_driven_density_functional_design_unformatted.pdf
https://github.com/deepmind/deepmind-research/tree/master/density_functional_approximation_dm21

GoogleAI trains GLaM: 1 Trillion Parameters Mixture of Experts Model
https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html

Colin Raffel calls for building ML models like we build Open-Source software
https://colinraffel.com/blog/a-call-to-build-models-like-we-build-open-source-software.html

A rebuke of the hype around DeepMind's math paper
https://arxiv.org/abs/2112.04324?s=09

Helpful Things
https://twitter.com/huggingface/status/1468996110207401992
https://docs.cohere.ai/prompt-engineering-wiki/?utm_source=pocket_mylist
https://github.blog/2021-12-08-improving-github-code-search/
https://huggingface.co/blog/data-measurements-tool
https://huggingface.co/spaces/huggingface/data-measurements-tool
https://blogs.microsoft.com/ai-for-business/building-ai-responsibly-from-research-to-practice/
https://techcommunity.microsoft.com/t5/azure-ai-blog/responsible-ai-dashboard-a-one-stop-shop-for-operationalizing/ba-p/3030944
https://github.com/minitorch/minitorch?utm_source=pocket_mylist
https://minitorch.github.io/
https://pandastutor.com/
https://pandastutor.com/vis.html
https://github.com/IAmPara0x/yuno
https://colab.research.google.com/drive/1WAewYgHDmDEWhPBBOvGgyLTiOaasVyOz?usp=sharing#scrollTo=hZamByTeBv3G
https://www.reddit.com/r/MachineLearning/comments/rbue4h/n_us_gov_launches_ml_competition_to_predict_snow/
https://www.drivendata.org/competitions/86/competition-reclamation-snow-water-dev/
https://www.reddit.com/r/MachineLearning/comments/rdb1uw/p_utttai_alphazerolike_solution_for_playing/
https://www.uttt.ai/
https://arxiv.org/abs/2112.02721?utm_source=pocket_mylist
https://arxiv.org/pdf/2112.02721.pdf
https://github.com/GEM-benchmark/NL-Augmenter
https://www.reddit.com/r/MachineLearning/comments/rdfdcv/p_collection_of_33_psychology_related_datasets/?utm_source=pocket_mylist

Suicide Capsule plans AI to assess your mental state before use
https://www.swissinfo.ch/eng/sci-tech/sarco-suicide-capsule--passes-legal-review--in-switzerland/46966510

Synthesia raises 50M to develop AI avatars
https://techcrunch.com/2021/12/08/synthesia-raises-50m-to-leverage-synthetic-avatars-for-corporate-training-and-more/
https://www.synthesia.io/

Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF

SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n