René's URL Explorer Experiment


Title: DeepLearningExamples/TensorFlow2/LanguageModeling/BERT at master · elscript/DeepLearningExamples · GitHub

Open Graph Title: DeepLearningExamples/TensorFlow2/LanguageModeling/BERT at master · elscript/DeepLearningExamples

X Title: DeepLearningExamples/TensorFlow2/LanguageModeling/BERT at master · elscript/DeepLearningExamples

Description: State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure. - DeepLearningExamples/TensorFlow2/LanguageModeling/BERT at master · elscript/DeepLearningExamples

Open Graph Description: State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure. - elscript/DeepLearningExamples

X Description: State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure. - elscript/DeepLearningExamples

Opengraph URL: https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/tree/*name(/*path)
route-controllerfiles
route-actiondisambiguate
fetch-noncev2:a30ec3ce-e9e1-c447-962b-75386aa3f8ef
current-catalog-service-hashf3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb
request-idA54E:317CD1:840C8E:BB243D:6A6212BD
html-safe-nonce5f1f726df06cb60f621187f7563a8bfebea3dc1b0317b029a424747fbf6bb744
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBNTRFOjMxN0NEMTo4NDBDOEU6QkIyNDNEOjZBNjIxMkJEIiwidmlzaXRvcl9pZCI6IjQ3MjUyMzkwNDU2NDM0NDA4MjkiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac2d8d6d59af8aa18924ab107170e750bb1bf8f1fb7730ff2ecf5ca57671eabe5c
hovercard-subject-tagrepository:870852723
github-keyboard-shortcutsrepository,source-code,file-tree,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///files/disambiguate
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT
twitter:imagehttps://opengraph.githubassets.com/ad0682913092d815eb1cae58e9c90cebcc58b9253aed2554ed63e3bb11f2c31f/elscript/DeepLearningExamples
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/ad0682913092d815eb1cae58e9c90cebcc58b9253aed2554ed63e3bb11f2c31f/elscript/DeepLearningExamples
og:image:altState-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure. - elscript/DeepLearningExamples
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Noneccf61cb7ab435725b916ed640f57c45b2c4721db9469bc32656080efc5214b3e
turbo-cache-controlno-cache
go-importgithub.com/elscript/DeepLearningExamples git https://github.com/elscript/DeepLearningExamples.git
octolytics-dimension-user_id7715754
octolytics-dimension-user_loginelscript
octolytics-dimension-repository_id870852723
octolytics-dimension-repository_nwoelscript/DeepLearningExamples
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forktrue
octolytics-dimension-repository_parent_id131881622
octolytics-dimension-repository_parent_nwoNVIDIA/DeepLearningExamples
octolytics-dimension-repository_network_root_id131881622
octolytics-dimension-repository_network_root_nwoNVIDIA/DeepLearningExamples
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releasea7585b3d2ee639495a70ce464d0a18bad88ea12c
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Felscript%2FDeepLearningExamples%2Ftree%2Fmaster%2FTensorFlow2%2FLanguageModeling%2FBERT
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub Copilot appDirect agents from issue to mergehttps://github.com/features/ai/github-app
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
Code QualityEnforce quality at mergehttps://github.com/features/code-quality
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
View all resourceshttps://github.com/resources
GitHub SponsorsFund open source developershttps://github.com/open-source/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/open-source/accelerator
GitHub Starshttps://stars.github.com
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Felscript%2FDeepLearningExamples%2Ftree%2Fmaster%2FTensorFlow2%2FLanguageModeling%2FBERT
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Ffiles%2Fdisambiguate&source=header-repo&source_repo=elscript%2FDeepLearningExamples
Reloadhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT
Reloadhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT
Reloadhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT
elscript https://github.com/elscript
DeepLearningExampleshttps://github.com/elscript/DeepLearningExamples
NVIDIA/DeepLearningExampleshttps://github.com/NVIDIA/DeepLearningExamples
Notifications https://github.com/login?return_to=%2Felscript%2FDeepLearningExamples
Fork 0 https://github.com/login?return_to=%2Felscript%2FDeepLearningExamples
Star 0 https://github.com/login?return_to=%2Felscript%2FDeepLearningExamples
Code https://github.com/elscript/DeepLearningExamples
Pull requests 0 https://github.com/elscript/DeepLearningExamples/pulls
Actions https://github.com/elscript/DeepLearningExamples/actions
Projects https://github.com/elscript/DeepLearningExamples/projects
Security and quality 0 https://github.com/elscript/DeepLearningExamples/security
Insights https://github.com/elscript/DeepLearningExamples/pulse
Code https://github.com/elscript/DeepLearningExamples
Pull requests https://github.com/elscript/DeepLearningExamples/pulls
Actions https://github.com/elscript/DeepLearningExamples/actions
Projects https://github.com/elscript/DeepLearningExamples/projects
Security and quality https://github.com/elscript/DeepLearningExamples/security
Insights https://github.com/elscript/DeepLearningExamples/pulse
https://github.com/elscript/DeepLearningExamples
DeepLearningExampleshttps://github.com/elscript/DeepLearningExamples/tree/master
TensorFlow2https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2
LanguageModelinghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling
Historyhttps://github.com/elscript/DeepLearningExamples/commits/master/TensorFlow2/LanguageModeling/BERT
https://github.com/elscript/DeepLearningExamples/commits/master/TensorFlow2/LanguageModeling/BERT
DeepLearningExampleshttps://github.com/elscript/DeepLearningExamples/tree/master
TensorFlow2https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2
LanguageModelinghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling
..https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling
datahttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT/data
datahttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT/data
officialhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT/official
officialhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT/official
scriptshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT/scripts
scriptshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT/scripts
.dockerignorehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/.dockerignore
.dockerignorehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/.dockerignore
.gitignorehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/.gitignore
.gitignorehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/.gitignore
Dockerfilehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/Dockerfile
Dockerfilehttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/Dockerfile
README.mdhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/README.md
README.mdhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/README.md
classifier_data_lib.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/classifier_data_lib.py
classifier_data_lib.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/classifier_data_lib.py
common_flags.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/common_flags.py
common_flags.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/common_flags.py
create_finetuning_data.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/create_finetuning_data.py
create_finetuning_data.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/create_finetuning_data.py
create_pretraining_data.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/create_pretraining_data.py
create_pretraining_data.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/create_pretraining_data.py
dllogger_class.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/dllogger_class.py
dllogger_class.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/dllogger_class.py
gpu_affinity.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/gpu_affinity.py
gpu_affinity.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/gpu_affinity.py
input_pipeline.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/input_pipeline.py
input_pipeline.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/input_pipeline.py
model_saving_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/model_saving_utils.py
model_saving_utils.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/model_saving_utils.py
optimization.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/optimization.py
optimization.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/optimization.py
run.subhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/run.sub
run.subhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/run.sub
run_classifier.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/run_classifier.py
run_classifier.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/run_classifier.py
run_pretraining.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/run_pretraining.py
run_pretraining.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/run_pretraining.py
run_squad.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/run_squad.py
run_squad.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/run_squad.py
squad_lib.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/squad_lib.py
squad_lib.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/squad_lib.py
squad_lib_sp.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/squad_lib_sp.py
squad_lib_sp.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/squad_lib_sp.py
tf_trt.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/tf_trt.py
tf_trt.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/tf_trt.py
tokenization.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/tokenization.py
tokenization.pyhttps://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/tokenization.py
README.mdhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#readme
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#bert-10-for-tensorflow-2
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#table-of-contents
Model overviewhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#model-overview
Model architecturehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#model-architecture
Default configurationhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#default-configuration
Feature support matrixhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#feature-support-matrix
Featureshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#features
Mixed precision traininghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#mixed-precision-training
Enabling mixed precisionhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#enabling-mixed-precision
Enabling TF32https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#enabling-tf32
Glossaryhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#glossary
Setuphttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#setup
Requirementshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#requirements
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#quick-start-guide
Advancedhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#advanced
Scripts and sample codehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#scripts-and-sample-code
Command-line optionshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#command-line-options
Getting the datahttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#getting-the-data
Dataset guidelineshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#dataset-guidelines
Multi-datasethttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#multi-dataset
Training processhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#training-process
Pre-traininghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training
Fine tuninghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning
Multi-nodehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#multi-node
Inference processhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#inference-process
Performancehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#performance
Benchmarkinghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#benchmarking
Training performance benchmarkhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#training-performance-benchmark
Inference performance benchmarkhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#inference-performance-benchmark
Resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#results
Training accuracy resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#training-accuracy-results
Pre-training accuracyhttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-accuracy
Fine-tuning accuracy for SQuAD v1.1: NVIDIA DGX A100 (8x A100 40GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-accuracy-for-squad-v1.1-nvidia-dgx-a100-(8x-a100-40gb)
Pre-training SQuAD v1.1 stability test: NVIDIA DGX A100 (256x A100 80GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-squad-v1.1-stability-test-nvidia-dgx-a100-(256x-a100-80gb)
Fine-tuning SQuAD v1.1 stability test: NVIDIA DGX A100 (8x A100 80GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-squad-v1.1-stability-test-nvidia-dgx-a100-(8x-a100-80gb)
Training performance resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#training-performance-results
Pre-training training performance: Single-node on NVIDIA DGX-2 V100 (16x V100 32GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-training-performance-single-node-on-nvidia-dgx-1-v100-(16x-v100-32gb)
Pre-training training performance: Multi-node on NVIDIA DGX-2H V100 (16x V100 32GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-training-performance-multi-node-on-nvidia-dgx-2h-v100-(16x-v100-32gb)
Pre-training training performance: Single-node on NVIDIA DGX A100 (8x A100 80GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-training-performance-single-node-on-nvidia-dgx-a100-(8x-a100-80gb)
Pre-training training performance: Multi-node on NVIDIA DGX A100 (8x A100 40GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-training-performance-multi-node-on-nvidia-dgx-a100-(8x-a100-40gb)
Fine-tuning training performance for SQuAD v1.1 on NVIDIA DGX-1 V100 (8x V100 16GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-training-performance-for-squad-v1.1-on-nvidia-dgx-1-v100-(8x-v100-16gb)
Fine-tuning training performance for SQuAD v1.1 on NVIDIA DGX-1 V100 (8x V100 32GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-training-performance-for-squad-v1.1-on-nvidia-dgx-1-v100-(8x-v100-32gb)
Fine-tuning training performance for SQuAD v1.1 on NVIDIA DGX A100 (8x A100 80GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-training-performance-for-squad-v1.1-on-nvidia-dgx-a100-(8x-a100-80gb)
Inference performance resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#inference-performance-results
Fine-tuning inference performance for SQuAD v1.1 on NVIDIA DGX-1 V100 (1x V100 16GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-inference-performance-for-squad-v1.1-on-nvidia-dgx-1-v100-(1x-v100-16gb)
Fine-tuning inference performance for SQuAD v1.1 on NVIIDA DGX-1 V100 (1x V100 32GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-inference-performance-for-squad-v1.1-on-nvidia-dgx-1-v100-(1x-v100-32gb)
Fine-tuning inference performance for SQuAD v1.1 on NVIDIA DGX A100 (8x A100 80GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-inference-performance-for-squad-v1.1-on-nvidia-dgx-a100-(1x-a100-80gb)
Fine-tuning inference performance for SQuAD v1.1 on NVIDIA Tesla T4 (1x T4 16GB)https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-inference-performance-for-squad-v1.1-on-nvidia-tesla-t4-(1x-t4-16gb)
Release Noteshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#release-notes
Changeloghttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#changelog
Known issueshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#known-issues
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#model-overview
BERT: Pre-training of Deep Bidirectional Transformers for Language Understandinghttps://arxiv.org/abs/1810.04805
Google's official implementationhttps://github.com/google-research/bert
NVIDIA PyTorchhttps://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/LanguageModeling/BERT
Hugging Facehttps://github.com/huggingface/pytorch-pretrained-BERT
codertimohttps://github.com/codertimo/BERT-pytorch
gluon-nlphttps://github.com/dmlc/gluon-nlp/tree/master/scripts/bert
Google's official implementationhttps://github.com/google-research/bert
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#model-architecture
https://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/data/images/bert_pipeline.png?raw=true
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#default-configuration
Wikipediahttps://dumps.wikimedia.org/
BookCorpushttp://yknzhu.wixsite.com/mbweb
SQuADhttps://rajpurkar.github.io/SQuAD-explorer/
XLAhttps://www.tensorflow.org/xla
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#feature-support-matrix
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#features
TensorFlow tutorialhttps://github.com/horovod/horovod/#usage
LAMBhttps://arxiv.org/pdf/1904.00962.pdf
Adamhttps://arxiv.org/pdf/1412.6980.pdf
LAMB version 1https://arxiv.org/abs/1904.00962v1
articlehttps://medium.com/@NvidiaAI/a-guide-to-optimizer-implementation-for-bert-at-scale-8338cc7f45fd
https://github.com/elscript/DeepLearningExamples/blob/master/TensorFlow2/LanguageModeling/BERT/data/images/images_nvlamb.png
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#mixed-precision-training
Mixed precisionhttps://arxiv.org/abs/1710.03740
Tensor Coreshttps://developer.nvidia.com/tensor-cores
mixed precision methodologyhttps://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html#tensorflow
Mixed Precision Traininghttps://arxiv.org/abs/1710.03740
Training With Mixed Precisionhttps://docs.nvidia.com/deeplearning/performance/mixed-precision-training/index.html
Mixed Precision Training of Deep Neural Networkshttps://devblogs.nvidia.com/mixed-precision-training-deep-neural-networks/
Using TF-AMPhttps://docs.nvidia.com/deeplearning/dgx/tensorflow-user-guide/index.html#tfamp
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#enabling-mixed-precision
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#enabling-tf32
NVIDIA A100https://www.nvidia.com/en-us/data-center/a100/
TensorFloat-32 in the A100 GPU Accelerates AI Training, HPC up to 20xhttps://blogs.nvidia.com/blog/2020/05/14/tensorfloat-32-precision-format/
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#glossary
Attention Is All You Needhttps://arxiv.org/abs/1706.03762
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#setup
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#requirements
NVIDIA Dockerhttps://github.com/NVIDIA/nvidia-docker
TensorFlow 21.02-py3+https://ngc.nvidia.com/catalog/containers/nvidia:tensorflow
NVIDIA Voltahttps://www.nvidia.com/en-us/data-center/volta-gpu-architecture/
NVIDIA Turinghttps://www.nvidia.com/en-us/geforce/turing/
NVIDIA Ampere architecturehttps://www.nvidia.com/en-us/data-center/nvidia-ampere-gpu-architecture/
Getting Started Using NVIDIA GPU Cloudhttps://docs.nvidia.com/ngc/ngc-getting-started-guide/index.html
Accessing And Pulling From The NGC Container Registryhttps://docs.nvidia.com/deeplearning/frameworks/user-guide/index.html#accessing_registry
Running TensorFlowhttps://docs.nvidia.com/deeplearning/frameworks/tensorflow-release-notes/running.html#running
NVIDIA Container Support Matrixhttps://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html
Enroothttps://github.com/NVIDIA/enroot
Pyxishttps://github.com/NVIDIA/pyxis
SLURMhttps://slurm.schedmd.com
Multi-node Documentationhttps://docs.nvidia.com/ngc/multi-node-bert-user-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#quick-start-guide
NGC model cataloghttps://ngc.nvidia.com/catalog/models
LAMB paperhttps://arxiv.org/pdf/1904.00962.pdf
BERT paperhttps://arxiv.org/pdf/1810.04805.pdf
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#advanced
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#scripts-and-sample-code
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#parameters
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#command-line-options
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#getting-the-data
herehttps://dumps.wikimedia.your.org/enwiki/latest/enwiki-latest-pages-articles.xml.bz2
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#dataset-guidelines
WordPiece tokenizationhttps://arxiv.org/pdf/1609.08144.pdf
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#multi-dataset
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#training-process
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#multi-node
Requirementshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#requirements
Training performance resultshttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#training-performance-results
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#inference-process
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#performance
NVIDIA Data Center Deep Learning Product Performancehttps://developer.nvidia.com/deep-learning-performance-training-inference
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#benchmarking
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#training-performance-benchmark
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#inference-performance-benchmark
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#results
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#training-accuracy-results
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-accuracy
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-accuracy-for-squad-v11-nvidia-dgx-a100-8x-a100-40gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-squad-v11-stability-test-nvidia-dgx-a100-256x-a100-80gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-squad-v11-stability-test-nvidia-dgx-a100-8x-a100-80gb
https://ngc.nvidia.com/catalog/modelshttps://ngc.nvidia.com/catalog/models
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#training-performance-results
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-training-performance-single-node-on-nvidia-dgx-2-v100-16x-v100-32gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-training-performance-multi-node-on-nvidia-dgx-2h-v100-16x-v100-32gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-training-performance-single-node-on-nvidia-dgx-a100-8x-a100-80gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#pre-training-training-performance-multi-node-on-nvidia-dgx-a100-8x-a100-80gb
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-training-performance-for-squad-v11-on-nvidia-dgx-1-v100-8x-v100-16gb
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-training-performance-for-squad-v11-on-nvidia-dgx-1-v100-8x-v100-32gb
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-training-performance-for-squad-v11-on-nvidia-dgx-a100-8x-a100-80gb
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#inference-performance-results
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-inference-performance-for-squad-v11-on-nvidia-dgx-1-v100-1x-v100-16gb
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-inference-performance-for-squad-v11-on-nviida-dgx-1-v100-1x-v100-32gb
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-inference-performance-for-squad-v11-on-nvidia-dgx-a100-1x-a100-80gb
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#fine-tuning-inference-performance-for-squad-v11-on-nvidia-tesla-t4-1x-t4-16gb
Quick Start Guidehttps://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#quick-start-guide
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#release-notes
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#changelog
https://github.com/elscript/DeepLearningExamples/tree/master/TensorFlow2/LanguageModeling/BERT#known-issues
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.