René's URL Explorer Experiment


Title: DeepLearningExamples/TensorFlow2/LanguageModeling/ELECTRA at master · elscript/DeepLearningExamples · GitHub

Open Graph Title: DeepLearningExamples/TensorFlow2/LanguageModeling/ELECTRA at master · elscript/DeepLearningExamples

X Title: DeepLearningExamples/TensorFlow2/LanguageModeling/ELECTRA at master · elscript/DeepLearningExamples

Description: State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure. - DeepLearningExamples/TensorFlow2/LanguageModeling/ELECTRA at master · elscript/DeepLearningExamples

Open Graph Description: State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure. - elscript/DeepLearningExamples

X Description: State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure. - elscript/DeepLearningExamples

Opengraph URL: https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/tree/*name(/*path)
route-controllerfiles
route-actiondisambiguate
fetch-noncev2:cc5e8f5b-aa7a-3a9b-3c47-48a009c7b080
current-catalog-service-hashf3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb
request-id8D80:1ACEC2:55AC6D:7CD832:6A620E36
html-safe-nonceb187c842a23ebf943e43d258b58380702cf48dc87f30bd3b85f20dfd076143eb
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4RDgwOjFBQ0VDMjo1NUFDNkQ6N0NEODMyOjZBNjIwRTM2IiwidmlzaXRvcl9pZCI6IjQyMzE4MzM5MTQ1MjY5NjExOCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac8c3eb83c80996c6021321881b8f36ddb5c1e92f8d8663c9e5c65211bcdb2256e
hovercard-subject-tagrepository:870852723
github-keyboard-shortcutsrepository,source-code,file-tree,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///files/disambiguate
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA
twitter:imagehttps://opengraph.githubassets.com/ad0682913092d815eb1cae58e9c90cebcc58b9253aed2554ed63e3bb11f2c31f/elscript/DeepLearningExamples
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/ad0682913092d815eb1cae58e9c90cebcc58b9253aed2554ed63e3bb11f2c31f/elscript/DeepLearningExamples
og:image:altState-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure. - elscript/DeepLearningExamples
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Nonebe9a4d9589756e22b427637ffbf1123a96e067050e57f5cec403e3080a650ca4
turbo-cache-controlno-cache
go-importgithub.com/elscript/DeepLearningExamples git https://github.com/elscript/DeepLearningExamples.git
octolytics-dimension-user_id7715754
octolytics-dimension-user_loginelscript
octolytics-dimension-repository_id870852723
octolytics-dimension-repository_nwoelscript/DeepLearningExamples
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forktrue
octolytics-dimension-repository_parent_id131881622
octolytics-dimension-repository_parent_nwoNVIDIA/DeepLearningExamples
octolytics-dimension-repository_network_root_id131881622
octolytics-dimension-repository_network_root_nwoNVIDIA/DeepLearningExamples
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release7b5509702f01f876c214a2cbb142690847de0019
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Felscript%2FDeepLearningExamples%2Ftree%2Fmaster%2FTensorFlow2%2FLanguageModeling%2FELECTRA
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub Copilot appDirect agents from issue to mergehttps://github.com/features/ai/github-app
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
Code QualityEnforce quality at mergehttps://github.com/features/code-quality
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
View all resourceshttps://github.com/resources
GitHub SponsorsFund open source developershttps://github.com/open-source/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/open-source/accelerator
GitHub Starshttps://stars.github.com
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Felscript%2FDeepLearningExamples%2Ftree%2Fmaster%2FTensorFlow2%2FLanguageModeling%2FELECTRA
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Ffiles%2Fdisambiguate&source=header-repo&source_repo=elscript%2FDeepLearningExamples
Reloadhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA
Reloadhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA
Reloadhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA
elscript https://github.com/elscript
DeepLearningExampleshttps://github.com/elscript/DeepLearningExamples
NVIDIA/DeepLearningExampleshttps://github.com/NVIDIA/DeepLearningExamples
Notifications https://github.com/login?return_to=%2Felscript%2FDeepLearningExamples
Fork 0 https://github.com/login?return_to=%2Felscript%2FDeepLearningExamples
Star 0 https://github.com/login?return_to=%2Felscript%2FDeepLearningExamples
Code https://github.com/elscript/DeepLearningExamples
Pull requests 0 https://github.com/elscript/DeepLearningExamples/pulls
Actions https://github.com/elscript/DeepLearningExamples/actions
Projects https://github.com/elscript/DeepLearningExamples/projects
Security and quality 0 https://github.com/elscript/DeepLearningExamples/security
Insights https://github.com/elscript/DeepLearningExamples/pulse
Code https://github.com/elscript/DeepLearningExamples
Pull requests https://github.com/elscript/DeepLearningExamples/pulls
Actions https://github.com/elscript/DeepLearningExamples/actions
Projects https://github.com/elscript/DeepLearningExamples/projects
Security and quality https://github.com/elscript/DeepLearningExamples/security
Insights https://github.com/elscript/DeepLearningExamples/pulse
https://github.com/elscript/DeepLearningExamples
DeepLearningExampleshttps://github.com/elscript/DeepLearningExamples/tree/master
TensorFlow2https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2
LanguageModelinghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling
Historyhttps://github.com/elscript/DeepLearningExamples/commits/master/TensorFlow2/LanguageModeling/ELECTRA
https://github.com/elscript/DeepLearningExamples/commits/master/TensorFlow2/LanguageModeling/ELECTRA
DeepLearningExampleshttps://github.com/elscript/DeepLearningExamples/tree/master
TensorFlow2https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2
LanguageModelinghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling
..https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling
datahttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/data
datahttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/data
imageshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/images
imageshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/images
resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/results
resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/results
scriptshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/scripts
scriptshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/scripts
vocabhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/vocab
vocabhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA/vocab
.gitignorehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/.gitignore
.gitignorehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/.gitignore
Dockerfilehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/Dockerfile
Dockerfilehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/Dockerfile
LICENSEhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/LICENSE
LICENSEhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/LICENSE
NOTICEhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/NOTICE
NOTICEhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/NOTICE
README.mdhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/README.md
README.mdhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/README.md
build_pretraining_dataset.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/build_pretraining_dataset.py
build_pretraining_dataset.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/build_pretraining_dataset.py
configuration.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/configuration.py
configuration.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/configuration.py
configuration_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/configuration_utils.py
configuration_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/configuration_utils.py
file_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/file_utils.py
file_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/file_utils.py
gpu_affinity.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/gpu_affinity.py
gpu_affinity.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/gpu_affinity.py
modeling.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/modeling.py
modeling.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/modeling.py
modeling_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/modeling_utils.py
modeling_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/modeling_utils.py
optimization.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/optimization.py
optimization.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/optimization.py
postprocess_pretrained_ckpt.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/postprocess_pretrained_ckpt.py
postprocess_pretrained_ckpt.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/postprocess_pretrained_ckpt.py
pretrain_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/pretrain_utils.py
pretrain_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/pretrain_utils.py
run.subhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/run.sub
run.subhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/run.sub
run_inference.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/run_inference.py
run_inference.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/run_inference.py
run_pretraining.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/run_pretraining.py
run_pretraining.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/run_pretraining.py
run_tf_squad.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/run_tf_squad.py
run_tf_squad.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/run_tf_squad.py
squad_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/squad_utils.py
squad_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/squad_utils.py
tokenization.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/tokenization.py
tokenization.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/tokenization.py
tokenization_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/tokenization_utils.py
tokenization_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/tokenization_utils.py
utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/utils.py
utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/utils.py
README.mdhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#readme
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#electra-for-tensorflow2
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#table-of-contents
Model overviewhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#model-overview
Model architecturehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#model-architecture
Default configurationhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#default-configuration
Feature support matrixhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#feature-support-matrix
Featureshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#features
Mixed precision traininghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#mixed-precision-training
Enabling mixed precisionhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#enabling-mixed-precision
Enabling TF32https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#enabling-tf32
Glossaryhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#glossary
Setuphttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#setup
Requirementshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#requirements
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#quick-start-guide
Advancedhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#advanced
Scripts and sample codehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#scripts-and-sample-code
Parametershttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#parameters
Pre-training parametershttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-parameters
Fine-tuning parametershttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-parameters
Command-line optionshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#command-line-options
Getting the datahttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#getting-the-data
Multi-datasethttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#multi-dataset
Training processhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-process
Pre-traininghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training
Multi-nodehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#multi-node
Fine-tuninghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning
Inference processhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-process
Fine-tuning inferencehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-inference
Performancehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#performance
Benchmarkinghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#benchmarking
Training performance benchmarkhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-benchmark
Inference performance benchmarkhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-performance-benchmark
Resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#results
Training accuracy resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-accuracy-results
Pre-training loss curveshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-loss-curves
Pre-training loss resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-loss-results
Fine-tuning accuracy: NVIDIA DGX A100 (8x A100 40GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-accuracy-nvidia-dgx-a100-8x-a100-40gb
Fine-tuning accuracy: NVIDIA DGX-1 (8x V100 16GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-accuracy-nvidia-dgx-1-8x-v100-16gb
Fine-tuning accuracy: NVIDIA DGX-2 (16x V100 32GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-accuracy-nvidia-dgx-2-16x-v100-32gb
Training stability testhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-stability-test
Pre-training stability test: NVIDIA DGX A100 (8x A100 40GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-stability-test-nvidia-dgx-a100-8x-a100-40gb
Fine-tuning stability test: NVIDIA DGX-1 (8x V100 16GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-stability-test-nvidia-dgx-1-8x-v100-16gb
Training performance resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-results
Training performance: NVIDIA DGX A100 (8x A100 40GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-nvidia-dgx-a100-8x-a100-40gb
Pre-training NVIDIA DGX A100 (8x A100 40GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-nvidia-dgx-a100-8x-a100-40gb
Fine-tuning NVIDIA DGX A100 (8x A100 40GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-nvidia-dgx-a100-8x-a100-40gb
Training performance: NVIDIA DGX-1 (8x V100 16GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-nvidia-dgx-1-8x-v100-16gb
Pre-training NVIDIA DGX-1 (8x V100 16GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-nvidia-dgx-1-8x-v100-16gb
Fine-tuning NVIDIA DGX-1 (8x V100 16GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-nvidia-dgx-1-8x-v100-16gb
Training performance: NVIDIA DGX-2 (16x V100 32GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-nvidia-dgx-2-16x-v100-32gb
Pre-training NVIDIA DGX-2 (16x V100 32GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-nvidia-dgx-2-16x-v100-32gb
Fine-tuning NVIDIA DGX-2 (16x V100 32GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-nvidia-dgx-2-16x-v100-32gb
Inference performance resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-performance-results
Inference performance: NVIDIA DGX A100 (1x A100 40GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-performance-nvidia-dgx-a100-1x-a100-40gb
Fine-tuning inference on NVIDIA DGX A100 (1x A100 40GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-inference-on-nvidia-dgx-a100-1x-a100-40gb
Inference performance: NVIDIA T4https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-performance-nvidia-t4
Fine-tuning inference on NVIDIA T4https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-inference-on-nvidia-t4
Release noteshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#release-notes
Changeloghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#changelog
Known issueshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#known-issues
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#model-overview
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generatorshttps://openreview.net/forum?id=r1xMH1BtvB
Hugging Face implementationhttps://huggingface.co/transformers/model_doc/electra.html
Hugging Facehttps://huggingface.co/transformers/model_doc/electra.html
Google's implementationhttps://github.com/google-research/electra
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#model-architecture
https://camo.githubusercontent.com/4bfb9bae900d2f3a815268da363aa227ba75130226941467e5543e991b955ae4/68747470733a2f2f312e62702e626c6f6773706f742e636f6d2f2d734879626330336e4a526f2f586d664c6f6e67645659492f41414141414141414662492f61307435775f7a4f5a2d55747859616f516c566b6d54527379464a794664647451434c63424741735948512f73313630302f696d616765312e706e67
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#default-configuration
Google ELECTRA repositoryhttps://github.com/google-research/electra
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#feature-support-matrix
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#features
Horovod: Official repositoryhttps://github.com/horovod/horovod
TensorFlow tutorialhttps://github.com/horovod/horovod/#usage
AMPhttps://nvidia.github.io/apex/amp.html
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#mixed-precision-training
Mixed precisionhttps://arxiv.org/abs/1710.03740
Tensor Coreshttps://developer.nvidia.com/tensor-cores
mixed precision methodologyhttps://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html#tensorflow
Mixed Precision Traininghttps://arxiv.org/abs/1710.03740
Training With Mixed Precisionhttps://docs.nvidia.com/deeplearning/performance/mixed-precision-training/index.html
Mixed-Precision Training of Deep Neural Networkshttps://devblogs.nvidia.com/mixed-precision-training-deep-neural-networks/
Using TF-AMPhttps://docs.nvidia.com/deeplearning/dgx/tensorflow-user-guide/index.html#tfamp
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#enabling-mixed-precision
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#enabling-tf32
NVIDIA A100https://www.nvidia.com/en-us/data-center/a100/
TensorFloat-32 in the A100 GPU Accelerates AI Training, HPC up to 20xhttps://blogs.nvidia.com/blog/2020/05/14/tensorfloat-32-precision-format/
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#glossary
Attention Is All You Needhttps://arxiv.org/abs/1706.03762
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#setup
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#requirements
NVIDIA Dockerhttps://github.com/NVIDIA/nvidia-docker
TensorFlow2 20.07-py3 NGC container or laterhttps://ngc.nvidia.com/registry/nvidia-tensorflow
NVIDIA Volta architecturehttps://www.nvidia.com/en-us/data-center/volta-gpu-architecture/
NVIDIA Turing architecturehttps://www.nvidia.com/en-us/geforce/turing/
NVIDIA Ampere architecturehttps://www.nvidia.com/en-us/data-center/nvidia-ampere-gpu-architecture/
Getting Started Using NVIDIA GPU Cloudhttps://docs.nvidia.com/ngc/ngc-getting-started-guide/index.html
Accessing And Pulling From The NGC Container Registryhttps://docs.nvidia.com/deeplearning/dgx/user-guide/index.html#accessing_registry
Running TensorFlow2https://docs.nvidia.com/deeplearning/frameworks/tensorflow-release-notes/running.html#running
NVIDIA Container Support Matrixhttps://docs.nvidia.com/deeplearning/dgx/support-matrix/index.html
Enroothttps://github.com/NVIDIA/enroot
Pyxishttps://github.com/NVIDIA/pyxis
SLURMhttps://slurm.schedmd.com
Multi-node Documentationhttps://docs.nvidia.com/ngc/multi-node-bert-user-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#quick-start-guide
Advancedhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#advanced
SQuADhttps://rajpurkar.github.io/SQuAD-explorer/
Multi-nodehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#multi-node
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#advanced
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#scripts-and-sample-code
SQuADhttps://rajpurkar.github.io/SQuAD-explorer/
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#parameters
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-parameters
paperhttps://openreview.net/forum?id=r1xMH1BtvB
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-parameters
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#command-line-options
Parametershttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#parameters
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#getting-the-data
SQuADhttps://rajpurkar.github.io/SQuAD-explorer/
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#multi-dataset
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-process
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#multi-node
Requirementshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#requirements
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning
Horovodhttps://github.com/horovod/horovod
Parametershttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#parameters
Fine-tuning parametershttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-parameters
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-process
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-inference
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#performance
NVIDIA Data Center Deep Learning Product Performancehttps://developer.nvidia.com/deep-learning-performance-training-inference
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#benchmarking
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-benchmark
Training processhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-process
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-performance-benchmark
Inference processhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-process
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#results
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-accuracy-results
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-loss-curves
https://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/ELECTRA/images/total_loss.svg
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-loss-results
Google's ELECTRA++ base pretrained checkpointhttps://github.com/google-research/electra#released-models
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-accuracy-nvidia-dgx-a100-8x-a100-40gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-accuracy-nvidia-dgx-1-8x-v100-16gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-accuracy-nvidia-dgx-2-16x-v100-32gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-stability-test
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-stability-test-nvidia-dgx-a100-8x-a100-40gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-stability-test-nvidia-dgx-1-8x-v100-16gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-results
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-nvidia-dgx-a100-8x-a100-40gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-nvidia-dgx-a100-8x-a100-40gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-nvidia-dgx-a100-8x-a100-40gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-nvidia-dgx-1-8x-v100-16gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-nvidia-dgx-1-8x-v100-16gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-nvidia-dgx-1-8x-v100-16gb
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#training-performance-nvidia-dgx-2-16x-v100-32gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#pre-training-nvidia-dgx-2-16x-v100-32gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-nvidia-dgx-2-16x-v100-32gb
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-performance-results
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-performance-nvidia-dgx-a100-1x-a100-40gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-inference-on-nvidia-dgx-a100-1x-a100-40gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#inference-performance-nvidia-t4
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#fine-tuning-inference-on-nvidia-t4
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#release-notes
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#changelog
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/ELECTRA#known-issues
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.