René's URL Explorer Experiment


Title: GitHub - cffan/bert

Open Graph Title: GitHub - cffan/bert

X Title: GitHub - cffan/bert

Description: Contribute to cffan/bert development by creating an account on GitHub.

Open Graph Description: Contribute to cffan/bert development by creating an account on GitHub.

X Description: Contribute to cffan/bert development by creating an account on GitHub.

Opengraph URL: https://github.com/cffan/bert

X: @github

direct link

Domain: patch-diff.githubusercontent.com

route-pattern/:user_id/:repository
route-controllerfiles
route-actiondisambiguate
fetch-noncev2:78d8c013-7568-c07b-f304-c1b72c5334e6
current-catalog-service-hashf3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb
request-idE788:722E5:1175DF:17729B:698F5B01
html-safe-nonce8f8ba7d883d1edee34663ceb452db99ef72433df55715a9ad5b0b91464502790
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFNzg4OjcyMkU1OjExNzVERjoxNzcyOUI6Njk4RjVCMDEiLCJ2aXNpdG9yX2lkIjoiMjc0MjMyMzYwMjIyMTEyODQ0OSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacb7089b5bfa82db5105f7674b9a8c6ea749245a95b60e4bec5186e7ee789dbeab
hovercard-subject-tagrepository:175758386
github-keyboard-shortcutsrepository,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location//
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/cffan/bert
twitter:imagehttps://opengraph.githubassets.com/6b298c4dedb5bf7996bf76a259f799a68db3becf6f8cf5a6c9a94c47ff39c418/cffan/bert
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/6b298c4dedb5bf7996bf76a259f799a68db3becf6f8cf5a6c9a94c47ff39c418/cffan/bert
og:image:altContribute to cffan/bert development by creating an account on GitHub.
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None974b952535e389ab3b5c0ef3f15bde9af29b6b38f7486069c43d99e51de0e563
turbo-cache-controlno-preview
go-importgithub.com/cffan/bert git https://github.com/cffan/bert.git
octolytics-dimension-user_id186422
octolytics-dimension-user_logincffan
octolytics-dimension-repository_id175758386
octolytics-dimension-repository_nwocffan/bert
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id175758386
octolytics-dimension-repository_network_root_nwocffan/bert
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
releaseb8e9d6dae5817b2dbe0a3d6d920ff8d3db837168
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://patch-diff.githubusercontent.com/cffan/bert#start-of-content
https://patch-diff.githubusercontent.com/
Sign in https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fcffan%2Fbert
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://patch-diff.githubusercontent.com/login?return_to=https%3A%2F%2Fgithub.com%2Fcffan%2Fbert
Sign up https://patch-diff.githubusercontent.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E&source=header-repo&source_repo=cffan%2Fbert
Reloadhttps://patch-diff.githubusercontent.com/cffan/bert
Reloadhttps://patch-diff.githubusercontent.com/cffan/bert
Reloadhttps://patch-diff.githubusercontent.com/cffan/bert
cffan https://patch-diff.githubusercontent.com/cffan
berthttps://patch-diff.githubusercontent.com/cffan/bert
Notifications https://patch-diff.githubusercontent.com/login?return_to=%2Fcffan%2Fbert
Fork 0 https://patch-diff.githubusercontent.com/login?return_to=%2Fcffan%2Fbert
Star 1 https://patch-diff.githubusercontent.com/login?return_to=%2Fcffan%2Fbert
Apache-2.0 license https://patch-diff.githubusercontent.com/cffan/bert/blob/master/LICENSE
1 star https://patch-diff.githubusercontent.com/cffan/bert/stargazers
0 forks https://patch-diff.githubusercontent.com/cffan/bert/forks
Branches https://patch-diff.githubusercontent.com/cffan/bert/branches
Tags https://patch-diff.githubusercontent.com/cffan/bert/tags
Activity https://patch-diff.githubusercontent.com/cffan/bert/activity
Star https://patch-diff.githubusercontent.com/login?return_to=%2Fcffan%2Fbert
Notifications https://patch-diff.githubusercontent.com/login?return_to=%2Fcffan%2Fbert
Code https://patch-diff.githubusercontent.com/cffan/bert
Issues 0 https://patch-diff.githubusercontent.com/cffan/bert/issues
Pull requests 0 https://patch-diff.githubusercontent.com/cffan/bert/pulls
Actions https://patch-diff.githubusercontent.com/cffan/bert/actions
Projects 0 https://patch-diff.githubusercontent.com/cffan/bert/projects
Security 0 https://patch-diff.githubusercontent.com/cffan/bert/security
Insights https://patch-diff.githubusercontent.com/cffan/bert/pulse
Code https://patch-diff.githubusercontent.com/cffan/bert
Issues https://patch-diff.githubusercontent.com/cffan/bert/issues
Pull requests https://patch-diff.githubusercontent.com/cffan/bert/pulls
Actions https://patch-diff.githubusercontent.com/cffan/bert/actions
Projects https://patch-diff.githubusercontent.com/cffan/bert/projects
Security https://patch-diff.githubusercontent.com/cffan/bert/security
Insights https://patch-diff.githubusercontent.com/cffan/bert/pulse
Brancheshttps://patch-diff.githubusercontent.com/cffan/bert/branches
Tagshttps://patch-diff.githubusercontent.com/cffan/bert/tags
https://patch-diff.githubusercontent.com/cffan/bert/branches
https://patch-diff.githubusercontent.com/cffan/bert/tags
1 Commithttps://patch-diff.githubusercontent.com/cffan/bert/commits/master/
https://patch-diff.githubusercontent.com/cffan/bert/commits/master/
.circlecihttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/.circleci
.circlecihttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/.circleci
dockerhttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/docker
dockerhttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/docker
exampleshttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/examples
exampleshttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/examples
notebookshttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/notebooks
notebookshttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/notebooks
pytorch_pretrained_berthttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/pytorch_pretrained_bert
pytorch_pretrained_berthttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/pytorch_pretrained_bert
sampleshttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/samples
sampleshttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/samples
testshttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/tests
testshttps://patch-diff.githubusercontent.com/cffan/bert/tree/master/tests
.gitignorehttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/.gitignore
.gitignorehttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/.gitignore
LICENSEhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/LICENSE
LICENSEhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/LICENSE
MANIFEST.inhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/MANIFEST.in
MANIFEST.inhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/MANIFEST.in
README.mdhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/README.md
README.mdhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/README.md
quant_aware_train_linear_quant.yamlhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/quant_aware_train_linear_quant.yaml
quant_aware_train_linear_quant.yamlhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/quant_aware_train_linear_quant.yaml
requirements.txthttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/requirements.txt
requirements.txthttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/requirements.txt
setup.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/setup.py
setup.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/setup.py
small_wiki_sentence_corpus.txthttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/small_wiki_sentence_corpus.txt
small_wiki_sentence_corpus.txthttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/small_wiki_sentence_corpus.txt
READMEhttps://patch-diff.githubusercontent.com/cffan/bert
Apache-2.0 licensehttps://patch-diff.githubusercontent.com/cffan/bert
https://patch-diff.githubusercontent.com/cffan/bert#pytorch-pretrained-bert-the-big--extending-repository-of-pretrained-transformers
https://circleci.com/gh/huggingface/pytorch-pretrained-BERT
Google's BERT modelhttps://github.com/google-research/bert
OpenAI's GPT modelhttps://github.com/openai/finetune-transformer-lm
Google/CMU's Transformer-XL modelhttps://github.com/kimiyoung/transformer-xl
OpenAI's GPT-2 modelhttps://blog.openai.com/better-language-models/
Exampleshttps://patch-diff.githubusercontent.com/cffan/bert#examples
BERT: Pre-training of Deep Bidirectional Transformers for Language Understandinghttps://arxiv.org/abs/1810.04805
Google's pre-trained modelshttps://github.com/google-research/bert
Improving Language Understanding by Generative Pre-Traininghttps://blog.openai.com/language-unsupervised/
PyTorch implementation by HuggingFacehttps://github.com/huggingface/pytorch-openai-transformer-lm
OpenAI's pre-trained modelhttps://github.com/openai/finetune-transformer-lm
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contexthttp://arxiv.org/abs/1901.02860
PyTorch implementationhttps://github.com/kimiyoung/transformer-xl
Language Models are Unsupervised Multitask Learnershttps://blog.openai.com/better-language-models/
OpenAI's implementationhttps://github.com/openai/gpt-2
OpenAI's pre-trained modelhttps://github.com/openai/gpt-2
https://patch-diff.githubusercontent.com/cffan/bert#content
Installationhttps://patch-diff.githubusercontent.com/cffan/bert#installation
Overviewhttps://patch-diff.githubusercontent.com/cffan/bert#overview
Usagehttps://patch-diff.githubusercontent.com/cffan/bert#usage
Dochttps://patch-diff.githubusercontent.com/cffan/bert#doc
Exampleshttps://patch-diff.githubusercontent.com/cffan/bert#examples
Notebookshttps://patch-diff.githubusercontent.com/cffan/bert#notebooks
TPUhttps://patch-diff.githubusercontent.com/cffan/bert#tpu
Command-line interfacehttps://patch-diff.githubusercontent.com/cffan/bert#Command-line-interface
https://patch-diff.githubusercontent.com/cffan/bert#installation
https://patch-diff.githubusercontent.com/cffan/bert#with-pip
https://patch-diff.githubusercontent.com/cffan/bert#from-source
tests folderhttps://github.com/huggingface/pytorch-pretrained-BERT/tree/master/tests
https://patch-diff.githubusercontent.com/cffan/bert#overview
Dochttps://patch-diff.githubusercontent.com/cffan/bert#doc
modeling.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py
BertModelhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py#L556
BertForMaskedLMhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py#L710
BertForNextSentencePredictionhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py#L771
BertForPreTraininghttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py#L639
BertForSequenceClassificationhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py#L833
BertForMultipleChoicehttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py#L899
BertForTokenClassificationhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py#L969
BertForQuestionAnsweringhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py#L1034
modeling_openai.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_openai.py
OpenAIGPTModelhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_openai.py#L537
OpenAIGPTLMHeadModelhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_openai.py#L691
OpenAIGPTDoubleHeadsModelhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_openai.py#L752
modeling_transfo_xl.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_transfo_xl.py
TransfoXLModelhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_transfo_xl.py#L974
TransfoXLLMHeadModelhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_transfo_xl.py#L1236
modeling_gpt2.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_gpt2.py
GPT2Modelhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_gpt2.py#L537
GPT2LMHeadModelhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_gpt2.py#L691
GPT2DoubleHeadsModelhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_gpt2.py#L752
tokenization.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/tokenization.py
tokenization_openai.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/tokenization_openai.py
tokenization_transfo_xl.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/tokenization_transfo_xl.py
tokenization_gpt2.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/tokenization_gpt2.py
optimization.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/optimization.py
optimization_openai.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/optimization_openai.py
modeling.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py
modeling_openai.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_openai.py
modeling_transfo_xl.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_transfo_xl.py
examples folderhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples
extract_features.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/extract_features.py
run_classifier.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_classifier.py
run_squad.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_squad.py
run_swag.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_swag.py
run_lm_finetuning.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_lm_finetuning.py
examples folderhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples
run_openai_gpt.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_openai_gpt.py
examples folderhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples
run_transfo_xl.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_transfo_xl.py
examples folderhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples
run_gpt2.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_gpt2.py
Exampleshttps://patch-diff.githubusercontent.com/cffan/bert#examples
notebooks folderhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/notebooks
Comparing-TF-and-PT-models.ipynbhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/notebooks/Comparing-TF-and-PT-models.ipynb
Comparing-TF-and-PT-models-SQuAD.ipynbhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/notebooks/Comparing-TF-and-PT-models-SQuAD.ipynb
Comparing-TF-and-PT-models-MLM-NSP.ipynbhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/notebooks/Comparing-TF-and-PT-models-MLM-NSP.ipynb
Notebookshttps://patch-diff.githubusercontent.com/cffan/bert#notebooks
Command-line interfacehttps://patch-diff.githubusercontent.com/cffan/bert#Command-line-interface
https://patch-diff.githubusercontent.com/cffan/bert#usage
https://patch-diff.githubusercontent.com/cffan/bert#bert
doc sectionhttps://patch-diff.githubusercontent.com/cffan/bert#doc
https://patch-diff.githubusercontent.com/cffan/bert#openai-gpt
doc sectionhttps://patch-diff.githubusercontent.com/cffan/bert#doc
https://patch-diff.githubusercontent.com/cffan/bert#transformer-xl
doc sectionhttps://patch-diff.githubusercontent.com/cffan/bert#doc
https://patch-diff.githubusercontent.com/cffan/bert#openai-gpt-2
doc sectionhttps://patch-diff.githubusercontent.com/cffan/bert#doc
https://patch-diff.githubusercontent.com/cffan/bert#doc
Loading Google AI's/OpenAI's pre-trained weigthshttps://patch-diff.githubusercontent.com/cffan/bert#loading-google-ai-or-openai-pre-trained-weigths-or-pytorch-dump
PyTorch modelshttps://patch-diff.githubusercontent.com/cffan/bert#PyTorch-models
Tokenizershttps://patch-diff.githubusercontent.com/cffan/bert#tokenizers
Optimizershttps://patch-diff.githubusercontent.com/cffan/bert#optimizerss
https://patch-diff.githubusercontent.com/cffan/bert#loading-google-ai-or-openai-pre-trained-weigths-or-pytorch-dump
herehttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py
Multilingual READMEhttps://github.com/google-research/bert/blob/master/multilingual.md
https://patch-diff.githubusercontent.com/cffan/bert#pytorch-models
https://patch-diff.githubusercontent.com/cffan/bert#1-bertmodel
modeling.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling.py
extract_features.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/extract_features.py
run_classifier.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_classifier.py
run_squad.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_squad.py
extract_features.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/extract_features.py
https://patch-diff.githubusercontent.com/cffan/bert#2-bertforpretraining
BertModelhttps://patch-diff.githubusercontent.com/cffan/bert#-1.-%60BertModel%60
run_lm_finetuning.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_lm_finetuning.py
https://patch-diff.githubusercontent.com/cffan/bert#3-bertformaskedlm
BertModelhttps://patch-diff.githubusercontent.com/cffan/bert#-1.-%60BertModel%60
https://patch-diff.githubusercontent.com/cffan/bert#4-bertfornextsentenceprediction
BertModelhttps://patch-diff.githubusercontent.com/cffan/bert#-1.-%60BertModel%60
https://patch-diff.githubusercontent.com/cffan/bert#5-bertforsequenceclassification
run_classifier.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_classifier.py
https://patch-diff.githubusercontent.com/cffan/bert#6-bertformultiplechoice
Improving Language Understanding by Generative Pre-Traininghttps://blog.openai.com/language-unsupervised/
issuehttps://github.com/google-research/bert/issues/38
run_swag.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_swag.py
https://patch-diff.githubusercontent.com/cffan/bert#7-bertfortokenclassification
https://patch-diff.githubusercontent.com/cffan/bert#8-bertforquestionanswering
run_squad.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_squad.py
https://patch-diff.githubusercontent.com/cffan/bert#9-openaigptmodel
modeling_openai.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_openai.py
https://patch-diff.githubusercontent.com/cffan/bert#10-openaigptlmheadmodel
OpenAIGPTModelhttps://patch-diff.githubusercontent.com/cffan/bert#-9.-%60OpenAIGPTModel%60
https://patch-diff.githubusercontent.com/cffan/bert#11-openaigptdoubleheadsmodel
OpenAIGPTModelhttps://patch-diff.githubusercontent.com/cffan/bert#-9.-%60OpenAIGPTModel%60
https://patch-diff.githubusercontent.com/cffan/bert#12-transfoxlmodel
modeling_transfo_xl.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_transfo_xl.py
https://patch-diff.githubusercontent.com/cffan/bert#extracting-a-list-of-the-hidden-states-at-each-layer-of-the-transformer-xl-from-last_hidden_state-and-new_mems
https://patch-diff.githubusercontent.com/cffan/bert#13-transfoxllmheadmodel
TransfoXLModelhttps://patch-diff.githubusercontent.com/cffan/bert#-12.-%60TransfoXLModel%60
https://patch-diff.githubusercontent.com/cffan/bert#14-gpt2model
modeling_gpt2.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/modeling_gpt2.py
https://patch-diff.githubusercontent.com/cffan/bert#15-gpt2lmheadmodel
GPT2Modelhttps://patch-diff.githubusercontent.com/cffan/bert#-14.-%60GPT2Model%60
https://patch-diff.githubusercontent.com/cffan/bert#16-gpt2doubleheadsmodel
GPT2Modelhttps://patch-diff.githubusercontent.com/cffan/bert#-14.-%60GPT2Model%60
https://patch-diff.githubusercontent.com/cffan/bert#tokenizers
https://patch-diff.githubusercontent.com/cffan/bert#berttokenizer
tokenization.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/tokenization.py
https://patch-diff.githubusercontent.com/cffan/bert#openaigpttokenizer
tokenization_openai.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/tokenization_openai.py
https://patch-diff.githubusercontent.com/cffan/bert#transfoxltokenizer
Efficient softmax approximation for GPUshttp://arxiv.org/abs/1609.04309
tokenization_transfo_xl.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/tokenization_transfo_xl.py
https://patch-diff.githubusercontent.com/cffan/bert#gpt2tokenizer
tokenization_gpt2.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/pytorch_pretrained_bert/tokenization_gpt2.py
https://patch-diff.githubusercontent.com/cffan/bert#optimizers
https://patch-diff.githubusercontent.com/cffan/bert#bertadam
https://patch-diff.githubusercontent.com/cffan/bert#openaigptadam
https://patch-diff.githubusercontent.com/cffan/bert#examples
Training large models: introduction, tools and exampleshttps://patch-diff.githubusercontent.com/cffan/bert#Training-large-models-introduction,-tools-and-examples
Fine-tuning with BERT: running the exampleshttps://patch-diff.githubusercontent.com/cffan/bert#Fine-tuning-with-BERT-running-the-examples
./exampleshttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples
Fine-tuning BERT-large on GPUshttps://patch-diff.githubusercontent.com/cffan/bert#Fine-tuning-BERT-large-on-GPUs
https://patch-diff.githubusercontent.com/cffan/bert#training-large-models-introduction-tools-and-examples
run_classifier.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_classifier.py
run_squad.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_squad.py
the tips on training large batches in PyTorchhttps://medium.com/huggingface/training-larger-batches-practical-tips-on-1-gpu-multi-gpu-distributed-setups-ec88c3e51255
herehttps://devblogs.nvidia.com/mixed-precision-training-deep-neural-networks/
herehttps://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html
as detailed herehttps://github.com/nvidia/apex
the doc and the associated repositoryhttps://github.com/nvidia/apex
the relevant PR of the present repositoryhttps://github.com/huggingface/pytorch-pretrained-BERT/pull/116
https://patch-diff.githubusercontent.com/cffan/bert#fine-tuning-with-bert-running-the-examples
the original implementationhttps://github.com/google-research/bert/
https://patch-diff.githubusercontent.com/cffan/bert#mrpc
GLUE datahttps://gluebenchmark.com/tasks
this scripthttps://gist.github.com/W4ngatang/60c2bdb54d156a41194446737ce03e2e
the original implementation hyper-parametershttps://github.com/google-research/bert#sentence-and-sentence-pair-classification-tasks
herehttps://github.com/NVIDIA/apex
https://patch-diff.githubusercontent.com/cffan/bert#squad
train-v1.1.jsonhttps://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json
dev-v1.1.jsonhttps://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json
evaluate-v1.1.pyhttps://github.com/allenai/bi-att-flow/blob/master/squad/evaluate-v1.1.py
https://patch-diff.githubusercontent.com/cffan/bert#swag
repositoryhttps://github.com/rowanz/swagaf
https://patch-diff.githubusercontent.com/cffan/bert#lm-fine-tuning
sample_text.txthttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/samples/sample_text.txt
exemplary training corpushttps://ext-bert-sample.obs.eu-de.otc.t-systems.com/small_wiki_sentence_corpus.txt
https://patch-diff.githubusercontent.com/cffan/bert#openai-gpt-transformer-xl-and-gpt-2-running-the-examples
https://patch-diff.githubusercontent.com/cffan/bert#fine-tuning-openai-gpt-on-the-rocstories-dataset
RocStories datasethttps://github.com/snigdhac/StoryComprehension_EMNLP/tree/master/Dataset/RoCStories
https://patch-diff.githubusercontent.com/cffan/bert#evaluating-the-pre-trained-transformer-xl-on-the-wikitext-103-dataset
https://patch-diff.githubusercontent.com/cffan/bert#unconditional-and-conditional-generation-from-openais-gpt-2-model
https://patch-diff.githubusercontent.com/cffan/bert#fine-tuning-bert-large-on-gpus
https://patch-diff.githubusercontent.com/cffan/bert#notebooks
three Jupyter Notebookshttps://github.com/huggingface/pytorch-pretrained-BERT/tree/master/notebooks
Comparing-TF-and-PT-models.ipynbhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/notebooks/Comparing-TF-and-PT-models.ipynb
Comparing-TF-and-PT-models-SQuAD.ipynbhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/notebooks/Comparing-TF-and-PT-models-SQuAD.ipynb
Comparing-TF-and-PT-models-MLM-NSP.ipynbhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/notebooks/Comparing-TF-and-PT-models-MLM-NSP.ipynb
https://patch-diff.githubusercontent.com/cffan/bert#command-line-interface
https://patch-diff.githubusercontent.com/cffan/bert#bert-1
the pre-trained models released by Googlehttps://github.com/google-research/bert#pre-trained-models
./pytorch_pretrained_bert/convert_tf_checkpoint_to_pytorch.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/convert_tf_checkpoint_to_pytorch.py
extract_features.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/extract_features.py
run_classifier.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/examples/run_classifier.py
run_squad.pyhttps://patch-diff.githubusercontent.com/cffan/bert/blob/master/(./examples/run_squad.py)
herehttps://github.com/google-research/bert#pre-trained-models
https://patch-diff.githubusercontent.com/cffan/bert#openai-gpt-1
herehttps://github.com/openai/finetune-transformer-lm
https://patch-diff.githubusercontent.com/cffan/bert#transformer-xl-1
herehttps://github.com/kimiyoung/transformer-xl/tree/master/tf#obtain-and-evaluate-pretrained-sota-models
https://patch-diff.githubusercontent.com/cffan/bert#gpt-2
https://patch-diff.githubusercontent.com/cffan/bert#tpu
official announcementhttps://cloud.google.com/blog/products/ai-machine-learning/introducing-pytorch-across-google-cloud
create_pretraining_data.pyhttps://github.com/google-research/bert/blob/master/create_pretraining_data.py
run_pretraining.pyhttps://github.com/google-research/bert/blob/master/run_pretraining.py
herehttps://github.com/google-research/bert#pre-training-with-bert
Readme https://patch-diff.githubusercontent.com/cffan/bert#readme-ov-file
Apache-2.0 license https://patch-diff.githubusercontent.com/cffan/bert#Apache-2.0-1-ov-file
Please reload this pagehttps://patch-diff.githubusercontent.com/cffan/bert
Activityhttps://patch-diff.githubusercontent.com/cffan/bert/activity
1 starhttps://patch-diff.githubusercontent.com/cffan/bert/stargazers
1 watchinghttps://patch-diff.githubusercontent.com/cffan/bert/watchers
0 forkshttps://patch-diff.githubusercontent.com/cffan/bert/forks
Report repository https://patch-diff.githubusercontent.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fcffan%2Fbert&report=cffan+%28user%29
Releaseshttps://patch-diff.githubusercontent.com/cffan/bert/releases
Packages 0https://patch-diff.githubusercontent.com/users/cffan/packages?repo_name=bert
Jupyter Notebook 55.1% https://patch-diff.githubusercontent.com/cffan/bert/search?l=jupyter-notebook
Python 44.9% https://patch-diff.githubusercontent.com/cffan/bert/search?l=python
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.