| route-pattern | /:user_id/:repository |
| route-controller | files |
| route-action | disambiguate |
| fetch-nonce | v2:0a09c900-beab-471b-d67d-cac001118c78 |
| current-catalog-service-hash | f3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb |
| request-id | D324:3304EF:4699558:5CF935D:698C5B89 |
| html-safe-nonce | 68ca54360fd9ad1325752c476e40a47e6245544aeff4b70df2ad648f3c446f7b |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEMzI0OjMzMDRFRjo0Njk5NTU4OjVDRjkzNUQ6Njk4QzVCODkiLCJ2aXNpdG9yX2lkIjoiMjY3Mzc2NTM1ODU1NzIyMzgxNyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 8253369fba47aee2a9c3aaae7ef2ecfe05d6dac9dac7c2bcb87218d27f10a1eb |
| hovercard-subject-tag | repository:645186413 |
| github-keyboard-shortcuts | repository,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | // |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/coliflowery/fairseq |
| twitter:image | https://opengraph.githubassets.com/620546a1152bf1259631860725575d6166ab2c3f33b96c0dca04c1b3967c8680/coliflowery/fairseq |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/620546a1152bf1259631860725575d6166ab2c3f33b96c0dca04c1b3967c8680/coliflowery/fairseq |
| og:image:alt | Facebook AI Research Sequence-to-Sequence Toolkit written in Python. - coliflowery/fairseq |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 640eeb7b6ff4d8d106235d228c0c286e82592d4d2403227b5b2b4fc5832297a4 |
| turbo-cache-control | no-preview |
| go-import | github.com/coliflowery/fairseq git https://github.com/coliflowery/fairseq.git |
| octolytics-dimension-user_id | 34336061 |
| octolytics-dimension-user_login | coliflowery |
| octolytics-dimension-repository_id | 645186413 |
| octolytics-dimension-repository_nwo | coliflowery/fairseq |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | true |
| octolytics-dimension-repository_parent_id | 101782647 |
| octolytics-dimension-repository_parent_nwo | facebookresearch/fairseq |
| octolytics-dimension-repository_network_root_id | 101782647 |
| octolytics-dimension-repository_network_root_nwo | facebookresearch/fairseq |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 3d444f0a47beeeac94cddbb51c91ab408befe8d4 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| Skip to content | https://patch-diff.githubusercontent.com/coliflowery/fairseq#start-of-content |
|
| https://patch-diff.githubusercontent.com/ |
|
Sign in
| https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fcoliflowery%2Ffairseq |
| GitHub CopilotWrite better code with AI | https://github.com/features/copilot |
| GitHub SparkBuild and deploy intelligent apps | https://github.com/features/spark |
| GitHub ModelsManage and compare prompts | https://github.com/features/models |
| MCP RegistryNewIntegrate external tools | https://github.com/mcp |
| ActionsAutomate any workflow | https://github.com/features/actions |
| CodespacesInstant dev environments | https://github.com/features/codespaces |
| IssuesPlan and track work | https://github.com/features/issues |
| Code ReviewManage code changes | https://github.com/features/code-review |
| GitHub Advanced SecurityFind and fix vulnerabilities | https://github.com/security/advanced-security |
| Code securitySecure your code as you build | https://github.com/security/advanced-security/code-security |
| Secret protectionStop leaks before they start | https://github.com/security/advanced-security/secret-protection |
| Why GitHub | https://github.com/why-github |
| Documentation | https://docs.github.com |
| Blog | https://github.blog |
| Changelog | https://github.blog/changelog |
| Marketplace | https://github.com/marketplace |
| View all features | https://github.com/features |
| Enterprises | https://github.com/enterprise |
| Small and medium teams | https://github.com/team |
| Startups | https://github.com/enterprise/startups |
| Nonprofits | https://github.com/solutions/industry/nonprofits |
| App Modernization | https://github.com/solutions/use-case/app-modernization |
| DevSecOps | https://github.com/solutions/use-case/devsecops |
| DevOps | https://github.com/solutions/use-case/devops |
| CI/CD | https://github.com/solutions/use-case/ci-cd |
| View all use cases | https://github.com/solutions/use-case |
| Healthcare | https://github.com/solutions/industry/healthcare |
| Financial services | https://github.com/solutions/industry/financial-services |
| Manufacturing | https://github.com/solutions/industry/manufacturing |
| Government | https://github.com/solutions/industry/government |
| View all industries | https://github.com/solutions/industry |
| View all solutions | https://github.com/solutions |
| AI | https://github.com/resources/articles?topic=ai |
| Software Development | https://github.com/resources/articles?topic=software-development |
| DevOps | https://github.com/resources/articles?topic=devops |
| Security | https://github.com/resources/articles?topic=security |
| View all topics | https://github.com/resources/articles |
| Customer stories | https://github.com/customer-stories |
| Events & webinars | https://github.com/resources/events |
| Ebooks & reports | https://github.com/resources/whitepapers |
| Business insights | https://github.com/solutions/executive-insights |
| GitHub Skills | https://skills.github.com |
| Documentation | https://docs.github.com |
| Customer support | https://support.github.com |
| Community forum | https://github.com/orgs/community/discussions |
| Trust center | https://github.com/trust-center |
| Partners | https://github.com/partners |
| GitHub SponsorsFund open source developers | https://github.com/sponsors |
| Security Lab | https://securitylab.github.com |
| Maintainer Community | https://maintainers.github.com |
| Accelerator | https://github.com/accelerator |
| Archive Program | https://archiveprogram.github.com |
| Topics | https://github.com/topics |
| Trending | https://github.com/trending |
| Collections | https://github.com/collections |
| Enterprise platformAI-powered developer platform | https://github.com/enterprise |
| GitHub Advanced SecurityEnterprise-grade security features | https://github.com/security/advanced-security |
| Copilot for BusinessEnterprise-grade AI features | https://github.com/features/copilot/copilot-business |
| Premium SupportEnterprise-grade 24/7 support | https://github.com/premium-support |
| Pricing | https://github.com/pricing |
| Search syntax tips | https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax |
| documentation | https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax |
|
Sign in
| https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fcoliflowery%2Ffairseq |
|
Sign up
| https://patch-diff.githubusercontent.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E&source=header-repo&source_repo=coliflowery%2Ffairseq |
| Reload | https://patch-diff.githubusercontent.com/coliflowery/fairseq |
| Reload | https://patch-diff.githubusercontent.com/coliflowery/fairseq |
| Reload | https://patch-diff.githubusercontent.com/coliflowery/fairseq |
|
coliflowery
| https://patch-diff.githubusercontent.com/coliflowery |
| fairseq | https://patch-diff.githubusercontent.com/coliflowery/fairseq |
| facebookresearch/fairseq | https://patch-diff.githubusercontent.com/facebookresearch/fairseq |
|
Notifications
| https://patch-diff.githubusercontent.com/login?return_to=%2Fcoliflowery%2Ffairseq |
|
Fork
0
| https://patch-diff.githubusercontent.com/login?return_to=%2Fcoliflowery%2Ffairseq |
|
Star
0
| https://patch-diff.githubusercontent.com/login?return_to=%2Fcoliflowery%2Ffairseq |
|
MIT license
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/LICENSE |
|
0
stars
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/stargazers |
|
6.7k
forks
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/forks |
|
Branches
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/branches |
|
Tags
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/tags |
|
Activity
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/activity |
|
Star
| https://patch-diff.githubusercontent.com/login?return_to=%2Fcoliflowery%2Ffairseq |
|
Notifications
| https://patch-diff.githubusercontent.com/login?return_to=%2Fcoliflowery%2Ffairseq |
|
Code
| https://patch-diff.githubusercontent.com/coliflowery/fairseq |
|
Pull requests
0
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/pulls |
|
Actions
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/actions |
|
Projects
0
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/projects |
|
Security
0
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/security |
|
Insights
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/pulse |
|
Code
| https://patch-diff.githubusercontent.com/coliflowery/fairseq |
|
Pull requests
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/pulls |
|
Actions
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/actions |
|
Projects
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/projects |
|
Security
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/security |
|
Insights
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/pulse |
| Branches | https://patch-diff.githubusercontent.com/coliflowery/fairseq/branches |
| Tags | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tags |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/branches |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/tags |
| 2,299 Commits | https://patch-diff.githubusercontent.com/coliflowery/fairseq/commits/main/ |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/commits/main/ |
| .circleci | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/.circleci |
| .circleci | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/.circleci |
| .github | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/.github |
| .github | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/.github |
| docs | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/docs |
| docs | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/docs |
| examples | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/examples |
| examples | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/examples |
| fairseq | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/fairseq |
| fairseq | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/fairseq |
| fairseq_cli | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/fairseq_cli |
| fairseq_cli | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/fairseq_cli |
| hydra_plugins/dependency_submitit_launcher | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/hydra_plugins/dependency_submitit_launcher |
| hydra_plugins/dependency_submitit_launcher | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/hydra_plugins/dependency_submitit_launcher |
| scripts | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/scripts |
| scripts | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/scripts |
| tests | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/tests |
| tests | https://patch-diff.githubusercontent.com/coliflowery/fairseq/tree/main/tests |
| .gitignore | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/.gitignore |
| .gitignore | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/.gitignore |
| .gitmodules | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/.gitmodules |
| .gitmodules | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/.gitmodules |
| .pre-commit-config.yaml | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/.pre-commit-config.yaml |
| .pre-commit-config.yaml | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/.pre-commit-config.yaml |
| CODE_OF_CONDUCT.md | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/CODE_OF_CONDUCT.md |
| CODE_OF_CONDUCT.md | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/CODE_OF_CONDUCT.md |
| CONTRIBUTING.md | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/CONTRIBUTING.md |
| CONTRIBUTING.md | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/CONTRIBUTING.md |
| LICENSE | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/LICENSE |
| LICENSE | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/LICENSE |
| MANIFEST.in | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/MANIFEST.in |
| MANIFEST.in | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/MANIFEST.in |
| README.md | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/README.md |
| README.md | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/README.md |
| RELEASE.md | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/RELEASE.md |
| RELEASE.md | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/RELEASE.md |
| hubconf.py | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/hubconf.py |
| hubconf.py | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/hubconf.py |
| pyproject.toml | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/pyproject.toml |
| pyproject.toml | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/pyproject.toml |
| release_utils.py | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/release_utils.py |
| release_utils.py | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/release_utils.py |
| setup.cfg | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/setup.cfg |
| setup.cfg | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/setup.cfg |
| setup.py | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/setup.py |
| setup.py | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/setup.py |
| train.py | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/train.py |
| train.py | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/train.py |
| README | https://patch-diff.githubusercontent.com/coliflowery/fairseq |
| Code of conduct | https://patch-diff.githubusercontent.com/coliflowery/fairseq |
| Contributing | https://patch-diff.githubusercontent.com/coliflowery/fairseq |
| License | https://patch-diff.githubusercontent.com/coliflowery/fairseq |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/docs/fairseq_logo.png |
| https://opensource.fb.com/support-ukraine |
| https://github.com/pytorch/fairseq/blob/main/LICENSE |
| https://github.com/pytorch/fairseq/releases |
| https://github.com/pytorch/fairseq/actions?query=workflow:build |
| https://fairseq.readthedocs.io/en/latest/?badge=latest |
| https://app.circleci.com/pipelines/github/facebookresearch/fairseq/ |
| Language Modeling with Gated Convolutional Networks (Dauphin et al., 2017) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/language_model/conv_lm/README.md |
| Convolutional Sequence to Sequence Learning (Gehring et al., 2017) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/conv_seq2seq/README.md |
| Classical Structured Prediction Losses for Sequence to Sequence Learning (Edunov et al., 2018) | https://github.com/pytorch/fairseq/tree/classic_seqlevel |
| Hierarchical Neural Story Generation (Fan et al., 2018) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/stories/README.md |
| wav2vec: Unsupervised Pre-training for Speech Recognition (Schneider et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/README.md |
| Pay Less Attention with Lightweight and Dynamic Convolutions (Wu et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/pay_less_attention_paper/README.md |
| Scaling Neural Machine Translation (Ott et al., 2018) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/scaling_nmt/README.md |
| Understanding Back-Translation at Scale (Edunov et al., 2018) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/backtranslation/README.md |
| Adaptive Input Representations for Neural Language Modeling (Baevski and Auli, 2018) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/language_model/README.adaptive_inputs.md |
| Lexically constrained decoding with dynamic beam allocation (Post & Vilar, 2018) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/constrained_decoding/README.md |
| Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context (Dai et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/truncated_bptt/README.md |
| Adaptive Attention Span in Transformers (Sukhbaatar et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/adaptive_span/README.md |
| Mixture Models for Diverse Machine Translation: Tricks of the Trade (Shen et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/translation_moe/README.md |
| RoBERTa: A Robustly Optimized BERT Pretraining Approach (Liu et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/roberta/README.md |
| Facebook FAIR's WMT19 News Translation Task Submission (Ng et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wmt19/README.md |
| Jointly Learning to Align and Translate with Transformer Models (Garg et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/joint_alignment_translation/README.md |
| Multilingual Denoising Pre-training for Neural Machine Translation (Liu et at., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/mbart/README.md |
| Neural Machine Translation with Byte-Level Subwords (Wang et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/byte_level_bpe/README.md |
| Unsupervised Quality Estimation for Neural Machine Translation (Fomicheva et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/unsupervised_quality_estimation/README.md |
| wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations (Baevski et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/README.md |
| Generating Medical Reports from Patient-Doctor Conversations Using Sequence-to-Sequence Models (Enarvi et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/pointer_generator/README.md |
| Linformer: Self-Attention with Linear Complexity (Wang et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/linformer/README.md |
| Cross-lingual Retrieval for Iterative Self-Supervised Training (Tran et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/criss/README.md |
| Deep Transformers with Latent Depth (Li et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/latent_depth/README.md |
| Unsupervised Cross-lingual Representation Learning for Speech Recognition (Conneau et al., 2020) | https://arxiv.org/abs/2006.13979 |
| Self-training and Pre-training are Complementary for Speech Recognition (Xu et al., 2020) | https://arxiv.org/abs/2010.11430 |
| Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training (Hsu, et al., 2021) | https://arxiv.org/abs/2104.01027 |
| Unsupervised Speech Recognition (Baevski, et al., 2021) | https://arxiv.org/abs/2105.11084 |
| Simple and Effective Zero-shot Cross-lingual Phoneme Recognition (Xu et al., 2021) | https://arxiv.org/abs/2109.11680 |
| VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding (Xu et. al., 2021) | https://arxiv.org/pdf/2109.14084.pdf |
| VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding (Xu et. al., 2021) | https://aclanthology.org/2021.findings-acl.370.pdf |
| NormFormer: Improved Transformer Pretraining with Extra Normalization (Shleifer et. al, 2021) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/normformer/README.md |
| Levenshtein Transformer (Gu et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/nonautoregressive_translation/README.md |
| Better Fine-Tuning by Reducing Representational Collapse (Aghajanyan et al. 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/rxf/README.md |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#whats-new |
| Released models for Scaling Speech Technology to 1,000+ Languages (Pratap, et al., 2023) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/mms/README.md |
| Released code for wav2vec-U 2.0 from Towards End-to-end Unsupervised Speech Recognition (Liu, et al., 2022) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/unsupervised/README.md |
| Integration with xFormers | https://github.com/facebookresearch/xformers |
| Released Direct speech-to-speech translation code | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/speech_to_speech/README.md |
| Released VideoCLIP and VLM models | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/MMPT/README.md |
| Released multilingual finetuned XLSR-53 model | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/README.md |
| master branch renamed to main | https://github.com/github/renaming |
| Released DrNMT code | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/discriminative_reranking_nmt/README.md |
| Released Robust wav2vec 2.0 model | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/README.md |
| Released XLMR-XL and XLMR-XXL models | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/xlmr/README.md |
| Released Unsupervised Speech Recognition code | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/unsupervised/README.md |
| Added full parameter and optimizer state sharding + CPU offloading | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/fully_sharded_data_parallel/README.md |
| Added LASER training code | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/laser/README.md |
| Added Adaptive Attention Span code | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/adaptive_span/README.md |
| GottBERT model and code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/gottbert/README.md |
| Hydra | https://github.com/facebookresearch/hydra |
| see documentation explaining how to use it for new and existing projects | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/docs/hydra_integration.md |
| fairseq 0.10.0 released | https://github.com/pytorch/fairseq/releases/tag/v0.10.0 |
| Added R3F/R4F (Better Fine-Tuning) code | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/rxf/README.md |
| Deep Transformer with Latent Depth code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/latent_depth/README.md |
| Added CRISS models and code | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/criss/README.md |
| Added Linformer code | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/linformer/README.md |
| Added pointer-generator networks | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/pointer_generator/README.md |
| Added lexically constrained decoding | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/constrained_decoding/README.md |
| wav2vec2 models and code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/README.md |
| Unsupervised Quality Estimation code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/unsupervised_quality_estimation/README.md |
| Follow fairseq on Twitter | https://twitter.com/fairseq |
| Monotonic Multihead Attention code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/simultaneous_translation/README.md |
| Quant-Noise code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/quant_noise/README.md |
| Initial model parallel support and 11B parameters unidirectional LM released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/megatron_11b/README.md |
| Byte-level BPE code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/byte_level_bpe/README.md |
| mBART model and code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/mbart/README.md |
| Added tutorial for back-translation | https://github.com/pytorch/fairseq/tree/main/examples/backtranslation#training-your-own-model-wmt18-english-german |
| fairseq 0.9.0 released | https://github.com/pytorch/fairseq/releases/tag/v0.9.0 |
| VizSeq released (a visual analysis toolkit for evaluating fairseq models) | https://facebookresearch.github.io/vizseq/docs/getting_started/fairseq_example |
| CamemBERT model and code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/camembert/README.md |
| BART model and code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/bart/README.md |
| XLM-R models and code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/xlmr/README.md |
| Nonautoregressive translation code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/nonautoregressive_translation/README.md |
| WMT'19 models released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wmt19/README.md |
| RoBERTa models and code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/roberta/README.md |
| wav2vec models and code released | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/README.md |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#features |
| Vijayakumar et al., 2016 | https://arxiv.org/abs/1610.02424 |
| lexically constrained decoding | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/constrained_decoding/README.md |
| gradient accumulation | https://fairseq.readthedocs.io/en/latest/getting_started.html#large-mini-batch-training-with-delayed-updates |
| mixed precision training | https://fairseq.readthedocs.io/en/latest/getting_started.html#training-with-half-precision-floating-point-fp16 |
| NVIDIA tensor cores | https://developer.nvidia.com/tensor-cores |
| extensible | https://fairseq.readthedocs.io/en/latest/overview.html |
| flexible configuration | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/docs/hydra_integration.md |
| Hydra | https://github.com/facebookresearch/hydra |
| full parameter and optimizer state sharding | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/fully_sharded_data_parallel/README.md |
| offloading parameters to CPU | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/fully_sharded_data_parallel/README.md |
| pre-trained models for translation and language modeling | https://patch-diff.githubusercontent.com/coliflowery/fairseq#pre-trained-models-and-examples |
| translation | https://pytorch.org/hub/pytorch_fairseq_translation/ |
| RoBERTa | https://pytorch.org/hub/pytorch_fairseq_roberta/ |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#requirements-and-installation |
| PyTorch | http://pytorch.org/ |
| NCCL | https://github.com/NVIDIA/nccl |
| apex | https://github.com/NVIDIA/apex |
| PyArrow | https://arrow.apache.org/docs/python/install.html#using-pip |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#getting-started |
| full documentation | https://fairseq.readthedocs.io/ |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#pre-trained-models-and-examples |
| Translation | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/translation/README.md |
| Language Modeling | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/language_model/README.md |
| XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale (Babu et al., 2021) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/xlsr/README.md |
| Cross-lingual Retrieval for Iterative Self-Supervised Training (Tran et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/criss/README.md |
| wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations (Baevski et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/README.md |
| Unsupervised Quality Estimation for Neural Machine Translation (Fomicheva et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/unsupervised_quality_estimation/README.md |
| Training with Quantization Noise for Extreme Model Compression ({Fan*, Stock*} et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/quant_noise/README.md |
| Neural Machine Translation with Byte-Level Subwords (Wang et al., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/byte_level_bpe/README.md |
| Multilingual Denoising Pre-training for Neural Machine Translation (Liu et at., 2020) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/mbart/README.md |
| Reducing Transformer Depth on Demand with Structured Dropout (Fan et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/layerdrop/README.md |
| Jointly Learning to Align and Translate with Transformer Models (Garg et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/joint_alignment_translation/README.md |
| Levenshtein Transformer (Gu et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/nonautoregressive_translation/README.md |
| Facebook FAIR's WMT19 News Translation Task Submission (Ng et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wmt19/README.md |
| RoBERTa: A Robustly Optimized BERT Pretraining Approach (Liu et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/roberta/README.md |
| wav2vec: Unsupervised Pre-training for Speech Recognition (Schneider et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/wav2vec/README.md |
| Mixture Models for Diverse Machine Translation: Tricks of the Trade (Shen et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/translation_moe/README.md |
| Pay Less Attention with Lightweight and Dynamic Convolutions (Wu et al., 2019) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/pay_less_attention_paper/README.md |
| Understanding Back-Translation at Scale (Edunov et al., 2018) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/backtranslation/README.md |
| Classical Structured Prediction Losses for Sequence to Sequence Learning (Edunov et al., 2018) | https://github.com/pytorch/fairseq/tree/classic_seqlevel |
| Hierarchical Neural Story Generation (Fan et al., 2018) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/stories/README.md |
| Scaling Neural Machine Translation (Ott et al., 2018) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/scaling_nmt/README.md |
| Convolutional Sequence to Sequence Learning (Gehring et al., 2017) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/conv_seq2seq/README.md |
| Language Modeling with Gated Convolutional Networks (Dauphin et al., 2017) | https://patch-diff.githubusercontent.com/coliflowery/fairseq/blob/main/examples/language_model/README.conv.md |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#join-the-fairseq-community |
| https://twitter.com/fairseq | https://twitter.com/fairseq |
| https://www.facebook.com/groups/fairseq.users | https://www.facebook.com/groups/fairseq.users |
| https://groups.google.com/forum/#!forum/fairseq-users | https://groups.google.com/forum/#!forum/fairseq-users |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#license |
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#citation |
|
Readme
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#readme-ov-file |
|
MIT license
| https://patch-diff.githubusercontent.com/coliflowery/fairseq#MIT-1-ov-file |
| Please reload this page | https://patch-diff.githubusercontent.com/coliflowery/fairseq |
|
Activity | https://patch-diff.githubusercontent.com/coliflowery/fairseq/activity |
|
0
stars | https://patch-diff.githubusercontent.com/coliflowery/fairseq/stargazers |
|
0
watching | https://patch-diff.githubusercontent.com/coliflowery/fairseq/watchers |
|
0
forks | https://patch-diff.githubusercontent.com/coliflowery/fairseq/forks |
|
Report repository
| https://patch-diff.githubusercontent.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fcoliflowery%2Ffairseq&report=coliflowery+%28user%29 |
| Releases | https://patch-diff.githubusercontent.com/coliflowery/fairseq/releases |
| Packages
0 | https://patch-diff.githubusercontent.com/users/coliflowery/packages?repo_name=fairseq |
|
| https://github.com |
| Terms | https://docs.github.com/site-policy/github-terms/github-terms-of-service |
| Privacy | https://docs.github.com/site-policy/privacy-policies/github-privacy-statement |
| Security | https://github.com/security |
| Status | https://www.githubstatus.com/ |
| Community | https://github.community/ |
| Docs | https://docs.github.com/ |
| Contact | https://support.github.com?tags=dotcom-footer |