Title: Posts
Domain: psc-g.github.io
| None | ie=edge |
Links:
| psc's website | https://psc-g.github.io/ |
| Posts | https://psc-g.github.io/posts |
| Mentoring / Education | https://psc-g.github.io/posts/mentoring/ |
| CME is A-OK | https://psc-g.github.io/posts/mentoring/cme/ |
| GridWorld Playground | https://psc-g.github.io/posts/mentoring/gridworldplayground/ |
| Intro a Transformers | https://psc-g.github.io/posts/mentoring/introduccion-a-transformers/ |
| Intro to RL | https://psc-g.github.io/posts/mentoring/intro-to-rl/ |
| Preparing your resume | https://psc-g.github.io/posts/mentoring/resume/ |
| Tips for Interviewing at Google | https://psc-g.github.io/posts/mentoring/interviewing/ |
| Tips for Reviewing Research Papers | https://psc-g.github.io/posts/mentoring/reviewing/ |
| MUSICODE | https://psc-g.github.io/posts/musicode/ |
| Phase 1 | https://psc-g.github.io/posts/musicode/phase1/ |
| 0-Introducing | https://psc-g.github.io/posts/musicode/phase1/introducing/ |
| 1-Musical Note & Computation | https://psc-g.github.io/posts/musicode/phase1/episode1/ |
| 2-Bits & Semitones | https://psc-g.github.io/posts/musicode/phase1/episode2/ |
| 3-Leitmotifs & Variables | https://psc-g.github.io/posts/musicode/phase1/episode3/ |
| 4-Live Coding & Jazz | https://psc-g.github.io/posts/musicode/phase1/episode4/ |
| 5-Repeats & Loops | https://psc-g.github.io/posts/musicode/phase1/episode5/ |
| Introducing | https://psc-g.github.io/posts/musicode/introducing/ |
| Losses, Dissonances, and Distortions | https://psc-g.github.io/posts/musicode/ldd/ |
| Portrait of Hallelagine | https://psc-g.github.io/posts/musicode/hallelagine/ |
| Art | https://psc-g.github.io/posts/art/ |
| Albums | https://psc-g.github.io/posts/art/albums/ |
| Cost of Beauty | https://psc-g.github.io/posts/art/cost-of-beauty/ |
| Covid Music | https://psc-g.github.io/posts/art/covid-music/ |
| Family | https://psc-g.github.io/posts/art/family/ |
| JiDiJi | https://psc-g.github.io/posts/art/jidiji/ |
| Musical Aquarium | https://psc-g.github.io/posts/art/musical-aquarium/ |
| Misc | https://psc-g.github.io/posts/misc/ |
| Artificial General Relativity | https://psc-g.github.io/posts/misc/agr/ |
| Crosswords | https://psc-g.github.io/posts/misc/crosswords/ |
| Origins of April Fool's Day | https://psc-g.github.io/posts/misc/origins/ |
| PongDay | https://psc-g.github.io/posts/misc/pongday/ |
| yovoy | https://psc-g.github.io/posts/misc/yovoy/ |
| Research | https://psc-g.github.io/posts/research/ |
| Other | https://psc-g.github.io/posts/research/other/ |
| RigL | https://psc-g.github.io/posts/research/other/rigl/rigl/ |
| RL | https://psc-g.github.io/posts/research/rl/ |
| 2020 RL Highlights | https://psc-g.github.io/posts/research/rl/2020highlights/ |
| Contrastive Behavioral Similarity Embeddings | https://psc-g.github.io/posts/research/rl/pse/ |
| Dopamine | https://psc-g.github.io/posts/research/rl/dopamine/ |
| Flying balloons with RL | https://psc-g.github.io/posts/research/rl/loon/ |
| From BBF to SSS | https://psc-g.github.io/posts/research/rl/from_bbf_to_sss/ |
| In defense of Atari | https://psc-g.github.io/posts/research/rl/atari_defense/ |
| Metrics & continuity in RL | https://psc-g.github.io/posts/research/rl/metrics_continuity/ |
| MICo | https://psc-g.github.io/posts/research/rl/mico/ |
| ReDo | https://psc-g.github.io/posts/research/rl/redo/ |
| Revisiting Rainbow | https://psc-g.github.io/posts/research/rl/revisiting_rainbow/ |
| Scalable methods ... | https://psc-g.github.io/posts/research/rl/scalable/ |
| SparseRL | https://psc-g.github.io/posts/research/rl/sparse_rl/ |
| Statistical Precipice | https://psc-g.github.io/posts/research/rl/precipice/ |
| Tandem RL | https://psc-g.github.io/posts/research/rl/tandem/ |
| Creativity | https://psc-g.github.io/posts/research/creativity/ |
| Agence, a dynamic film | https://psc-g.github.io/posts/research/creativity/agence/ |
| GANterpretations | https://psc-g.github.io/posts/research/creativity/ganterpretations/ |
| ML-Jam | https://psc-g.github.io/posts/research/creativity/ml-jam/ |
| In Defense of Atari - the ALE is not 'solved'!This post is based on a talk I gave at the AutoRL workshop in ICML 2024, which unfortunately was not recorded.IntroductionReinforcement Learning (RL) has been used successfully in a number of challenging tasks, such as beating world champions at Go, controlling tokamak plasmas for nuclear fusion, optimized chip placement, and controlling stratospheric balloons. All these successes have leveraged years of research and expertise and, importantly, rely on the combination of RL algorithms with deep neural networks (as proposed in the seminal DQN paper).December 2, 2024 Read | https://psc-g.github.io/posts/research/rl/atari_defense/ |
| AutoRL workshop in ICML 2024 | https://autorlworkshop.github.io/ |
| beating world champions at Go | https://deepmind.google/research/breakthroughs/alphago/ |
| controlling tokamak plasmas for nuclear fusion | https://www.nature.com/articles/s41586-021-04301-9 |
| optimized chip placement | https://www.nature.com/articles/s41586-021-03544-w |
| controlling stratospheric balloons | https://www.nature.com/articles/s41586-020-2939-8 |
| DQN paper | https://www.nature.com/articles/nature14236 |
| Read | https://psc-g.github.io/posts/research/rl/atari_defense/ |
| From "Bigger, Better, Faster" to "Smaller, Sparser, Stranger"This is a post based on a talk I gave a few times in 2023. I had been meaning to put it in blog post form for over a year but kept putting it off… I guess better late than never. I think some of the ideas still hold, so hope some of you find it useful!Bigger, better, fasterIn the seminal DQN paper, Mnih et al. demonstrated that reinforcement learning, when combined with neural networks as function approximators, could learn to play Atari 2600 games at superhuman levels. The DQN agent learned to do this over 200 million environment frames, which is roughly equivalent to 1000 hours of human gameplay…November 27, 2024 Read | https://psc-g.github.io/posts/research/rl/from_bbf_to_sss/ |
| seminal DQN paper | https://www.nature.com/articles/nature14236 |
| Read | https://psc-g.github.io/posts/research/rl/from_bbf_to_sss/ |
| The Dormant Neuron Phenomenon in Deep Reinforcement LearningWe identify the dormant neuron phenomenon in deep reinforcement learning, where an agent’s network suffers from an increasing number of inactive neurons, thereby affecting network expressivity.Ghada Sokar, Rishabh Agarwal, Pablo Samuel Castro*, Utku Evci*This blogpost is a summary of our ICML 2023 paper. The code is available here. Many more results and analyses are available in the paper, so I encouraged you to check it out if interested!The following figure gives a nice summary of the overall findings of our work (we are reporting the Interquantile Mean (IQM) as introduced in our Statistical Precipice NeurIPS'21 paper):June 19, 2023 Read | https://psc-g.github.io/posts/research/rl/redo/ |
| ICML 2023 paper | https://arxiv.org/abs/2302.12902 |
| here | https://github.com/google/dopamine/tree/master/dopamine/labs/redo |
| our Statistical Precipice NeurIPS'21 paper | https://arxiv.org/abs/2108.13264 |
| Read | https://psc-g.github.io/posts/research/rl/redo/ |
| AlbumsI’ve been in bands since I was 12. In a parallel universe I’m a full-time musician :D. This page collects the albums I’ve released so far, in reverse chronological order. Enjoy!the 45s - #1 (2024)With my good friend Jean-Simon Diallo we started a new project where we release 45s (i.e. singles) consisting of two songs, one written by him and one by me. He does the singing, I do the production and play the instruments. Enjoy!March 20, 2023 Read | https://psc-g.github.io/posts/art/albums/ |
| Read | https://psc-g.github.io/posts/art/albums/ |
| PongDayI learned on the radio that last November 29th marked the 50th anniversary of the classic arcade game Pong. This game is particularly meaningful for those of us that do RL research, as it is one of the games that is part of the Arcade Learning Environment, one of the most popular benchmarks. Pong is probably the easiest game of the whole suite, so we often use it as a test to make sure our agents are learning. Learning curves below are for agents trained with the Dopamine framework.December 5, 2022 Read | https://psc-g.github.io/posts/misc/pongday/ |
| Arcade Learning Environment | https://jair.org/index.php/jair/article/view/10819 |
| Dopamine | https://github.com/google/dopamine |
| Read | https://psc-g.github.io/posts/misc/pongday/ |
| Introducción a los TransformersComo parte de la RIIAA en Quito, di una introducción a los Transformers, que es la arquitectura detrás de avances como GPT-3, Music Transformer, Parti, y muchos otros.GrabaciónPueden ver la grabación aquí:MaterialesAquí pueden acceder a los diferentes materiales que mencioné durante el curso:Las diapositivas que usé en el cursoWrite with Transformers de Hugging Face (GPT-2)Eleuther GPT-J-6B, que es mucho mejor modelo que el GPT-2 de Hugging FaceEl colab simple sobre bigramsEl colab de Flax sobre LSTMsEl excelente the Illustrated Transformer de Jay Alammar, en el cual basé la descripción de Transformers.The Annotated TransformerEl blog post de PartiOctober 3, 2022 Read | https://psc-g.github.io/posts/mentoring/introduccion-a-transformers/ |
| RIIAA en Quito | https://www.riiaa.org/riiaa5-quito |
| Las diapositivas | https://docs.google.com/presentation/d/1nvUn5w1e01bzjFtuaZ50SZb773QEqxgwzlCa-X7XrfY/edit?usp=sharing |
| Write with Transformers | https://transformer.huggingface.co/ |
| Eleuther GPT-J-6B | https://6b.eleuther.ai/ |
| colab simple sobre bigrams | https://colab.research.google.com/drive/1psXc1nRmPCY6H7o7bBEY7ZvP68vZSVnN?usp=sharing |
| colab de Flax sobre LSTMs | https://colab.research.google.com/github/google/flax/blob/main/examples/seq2seq/seq2seq.ipynb |
| the Illustrated Transformer | https://jalammar.github.io/illustrated-transformer/ |
| The Annotated Transformer | http://nlp.seas.harvard.edu/annotated-transformer/ |
| blog post de Parti | https://parti.research.google/ |
| Read | https://psc-g.github.io/posts/mentoring/introduccion-a-transformers/ |
| Th State of Spars Train ng in D ep Re nforc m nt Le rn ngWe perform a systematic investigation into applying a number of existing sparse training techniques on a variety of deep RL agents and environments, and conclude by suggesting promising avenues for improving the effectiveness of sparse training methods, as well as for advancing their use in DRL.Laura Graesser*, Utku Evci*, Erich Elsen, Pablo Samuel CastroThis blogpost is a summary of our ICML 2022 paper. The code is available here. Many more results and analyses are available in the paper, so I encouraged you to check it out if interested!June 22, 2022 Read | https://psc-g.github.io/posts/research/rl/sparse_rl/ |
| ICML 2022 paper | https://arxiv.org/abs/2206.10369 |
| here | https://github.com/google-research/rigl/tree/master/rigl/rl |
| Read | https://psc-g.github.io/posts/research/rl/sparse_rl/ |
| Crosswords: A General Intelligence Challenge?I have become obsessed with crossword puzzles, specifically the NYT crosswords, since my friend Ralph Crewe gently forced me to start doing them. Although I’m not still at his level, I’ve been working on them daily and getting noticeably better.In doing so I’ve come to realize they are a fantastic mechanism for testing generally capable problem-solving, and in this post would like to explain the various types of challenges they present. I’ll be using past NYT crossword puzzles as examples (they’re all at least a week old so should hopefully not be spoilers for anyone).April 19, 2022 Read | https://psc-g.github.io/posts/misc/crosswords/ |
| the NYT crosswords | https://www.nytimes.com/crosswords |
| Read | https://psc-g.github.io/posts/misc/crosswords/ |
| yovoyWhat is a palindrome?A palindrome is a phrase that reads the same way from left to right, and right to left. The rules are that all characters must be used in both directions, but punctuation, capitalization, and spaces can be ignored.¡Las mismas reglas en español!Some well-known Palindromes:A man, a plan, a canal, Panama!Do geese see god?Yo, banana boy!Unos palíndromos en español:Dábale arroz a la zorra el abad.March 7, 2022 Read | https://psc-g.github.io/posts/misc/yovoy/ |
| palindrome | https://en.wikipedia.org/wiki/Palindrome |
| Read | https://psc-g.github.io/posts/misc/yovoy/ |
| CME is A-OKThe thread I wrote at the start of perf season at Google seemed to resonate with lots of people, so I decided to put a slightly extended version of it in blog-post form.What is perf?In brief, “perf” season at Google is when we evaluate our performance over the last few months, in the form of a self-assessment, and our peers provide their assessments on how they perceive our performance. The general purpose of this exercise is to receive feedback on how to grow as an engineer/researcher/employee, but it is also the process through which you can get promoted (by nominating yourself).February 28, 2022 Read | https://psc-g.github.io/posts/mentoring/cme/ |
| The thread I wrote | https://twitter.com/pcastr/status/1496344364389388289 |
| Read | https://psc-g.github.io/posts/mentoring/cme/ |
| Portrait of HallelagineHappy holidays from the MUSICODE “team”!“Portrait of Hallelagine”, a mashup of Jaco Pastorius’ “Portrait of Tracy”, Leonard Cohen’s “Hallelujah”, and John Lennon’s “Imagine”.100% of video editing done with Runway!December 23, 2021 Read | https://psc-g.github.io/posts/musicode/hallelagine/ |
| Runway | https://runwayml.com/ |
| Read | https://psc-g.github.io/posts/musicode/hallelagine/ |
| Deep Reinforcement Learning at the Edge of the Statistical PrecipiceWe argue that reliable evaluation in the few run deep RL regime cannot ignore the uncertainty in results without running the risk of slowing down progress in the field. We illustrate this point using a case study on the Atari 100k benchmark, where we find substantial discrepancies between conclusions drawn from point estimates alone versus a more thorough statistical analysis. We advocate for reporting interval estimates of aggregate performance and propose performance profiles to account for the variability in results, as well as present more robust and efficient aggregate metrics, such as interquartile mean scores, to achieve small uncertainty in results.December 6, 2021 Read | https://psc-g.github.io/posts/research/rl/precipice/ |
| Read | https://psc-g.github.io/posts/research/rl/precipice/ |
| 2 | https://psc-g.github.io/posts/page/2/ |
| 3 | https://psc-g.github.io/posts/page/3/ |
| 4 | https://psc-g.github.io/posts/page/4/ |
| » | https://psc-g.github.io/posts/page/2/ |
| »» | https://psc-g.github.io/posts/page/4/ |
| About | https://psc-g.github.io/posts#about |
| Recent Posts | https://psc-g.github.io/posts#recent-posts |
| Selected Publications | https://psc-g.github.io/posts#publications |
| Students | https://psc-g.github.io/posts#students |
| https://twitter.com/@pcastr | |
| Mastodon | https://sigmoid.social/@psc |
| Toha | https://github.com/hossainemruz/toha |
| Powered by | https://gohugo.io/ |
Viewport: width=device-width,initial-scale=1