René's URL Explorer Experiment


Title: Add recipe for audio/speech LLM (ltu-as with llama3) by BenoitWang · Pull Request #2550 · speechbrain/speechbrain · GitHub

Open Graph Title: Add recipe for audio/speech LLM (ltu-as with llama3) by BenoitWang · Pull Request #2550 · speechbrain/speechbrain

X Title: Add recipe for audio/speech LLM (ltu-as with llama3) by BenoitWang · Pull Request #2550 · speechbrain/speechbrain

Description: Hi @mravanelli, here's the ltu-as PR as discussed. I am collecting several new datasets and will start a new round of training but this may take time, so meanwhile I start this PR and carry on litt...

Open Graph Description: Hi @mravanelli, here's the ltu-as PR as discussed. I am collecting several new datasets and will start a new round of training but this may take time, so meanwhile I start this PR and carry on ...

X Description: Hi @mravanelli, here's the ltu-as PR as discussed. I am collecting several new datasets and will start a new round of training but this may take time, so meanwhile I start this PR and carry...

Opengraph URL: https://github.com/speechbrain/speechbrain/pull/2550

X: @github

direct link

Domain: github.com

route-pattern/_view_fragments/voltron/pull_requests/show/:user_id/:repository/:id/pull_request_layout(.:format)
route-controllervoltron_pull_requests_fragments
route-actionpull_request_layout
fetch-noncev2:4149561f-e935-47b4-d333-1744bf603225
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idEA46:27B2D:D8C40E:122800A:69649AD7
html-safe-nonce89ab2229c5e657c008d2c37a3e287f8f5dbd94b708a6e0aa2e5806767ec0a35a
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFQTQ2OjI3QjJEOkQ4QzQwRToxMjI4MDBBOjY5NjQ5QUQ3IiwidmlzaXRvcl9pZCI6IjYyODc2MTM5MTMwMDM3NjAzNDMiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac7768c2948383ca0f53d0cd0fe1a030776004de166fcca5ba0067094c82f3bd77
hovercard-subject-tagpull_request:1873300019
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/pull_requests_fragments/pull_request_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/voltron/pull_requests/show/speechbrain/speechbrain/2550/pull_request_layout
twitter:imagehttps://opengraph.githubassets.com/2c3508842dc118af606b4c210fd3f744ee6a398415da91b349531899fe64765c/speechbrain/speechbrain/pull/2550
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/2c3508842dc118af606b4c210fd3f744ee6a398415da91b349531899fe64765c/speechbrain/speechbrain/pull/2550
og:image:altHi @mravanelli, here's the ltu-as PR as discussed. I am collecting several new datasets and will start a new round of training but this may take time, so meanwhile I start this PR and carry on ...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameBenoitWang
hostnamegithub.com
expected-hostnamegithub.com
Nonebaa7d9900fdf7b27d604f36887af878d569cfbdcf97126832a5f4f0caf0c6ba5
turbo-cache-controlno-preview
go-importgithub.com/speechbrain/speechbrain git https://github.com/speechbrain/speechbrain.git
octolytics-dimension-user_id54749030
octolytics-dimension-user_loginspeechbrain
octolytics-dimension-repository_id259710503
octolytics-dimension-repository_nwospeechbrain/speechbrain
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id259710503
octolytics-dimension-repository_network_root_nwospeechbrain/speechbrain
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release842eff1d11f899d02b6b3b98fa3ea4860e64b34e
ui-targetcanary-2
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/speechbrain/speechbrain/pull/2550#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fspeechbrain%2Fspeechbrain%2Fpull%2F2550
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fspeechbrain%2Fspeechbrain%2Fpull%2F2550
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fpull_requests_fragments%2Fpull_request_layout&source=header-repo&source_repo=speechbrain%2Fspeechbrain
Reloadhttps://github.com/speechbrain/speechbrain/pull/2550
Reloadhttps://github.com/speechbrain/speechbrain/pull/2550
Reloadhttps://github.com/speechbrain/speechbrain/pull/2550
speechbrain https://github.com/speechbrain
speechbrainhttps://github.com/speechbrain/speechbrain
Notifications https://github.com/login?return_to=%2Fspeechbrain%2Fspeechbrain
Fork 1.6k https://github.com/login?return_to=%2Fspeechbrain%2Fspeechbrain
Star 11k https://github.com/login?return_to=%2Fspeechbrain%2Fspeechbrain
Code https://github.com/speechbrain/speechbrain
Issues 125 https://github.com/speechbrain/speechbrain/issues
Pull requests 50 https://github.com/speechbrain/speechbrain/pulls
Discussions https://github.com/speechbrain/speechbrain/discussions
Actions https://github.com/speechbrain/speechbrain/actions
Projects 0 https://github.com/speechbrain/speechbrain/projects
Security Uh oh! There was an error while loading. Please reload this page. https://github.com/speechbrain/speechbrain/security
Please reload this pagehttps://github.com/speechbrain/speechbrain/pull/2550
Insights https://github.com/speechbrain/speechbrain/pulse
Code https://github.com/speechbrain/speechbrain
Issues https://github.com/speechbrain/speechbrain/issues
Pull requests https://github.com/speechbrain/speechbrain/pulls
Discussions https://github.com/speechbrain/speechbrain/discussions
Actions https://github.com/speechbrain/speechbrain/actions
Projects https://github.com/speechbrain/speechbrain/projects
Security https://github.com/speechbrain/speechbrain/security
Insights https://github.com/speechbrain/speechbrain/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fspeechbrain%2Fspeechbrain%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fspeechbrain%2Fspeechbrain%2Fissues%2Fnew%2Fchoose
Jump to bottomhttps://github.com/speechbrain/speechbrain/pull/2550#issue-comment-box
BenoitWanghttps://github.com/BenoitWang
speechbrain:develophttps://github.com/speechbrain/speechbrain/tree/develop
BenoitWang:speech_llmhttps://github.com/BenoitWang/speechbrain/tree/speech_llm
Add recipe for audio/speech LLM (ltu-as with llama3) https://github.com/speechbrain/speechbrain/pull/2550#top
BenoitWanghttps://github.com/BenoitWang
speechbrain:develophttps://github.com/speechbrain/speechbrain/tree/develop
BenoitWang:speech_llmhttps://github.com/BenoitWang/speechbrain/tree/speech_llm
Conversation 1 https://github.com/speechbrain/speechbrain/pull/2550
Commits 26 https://github.com/speechbrain/speechbrain/pull/2550/commits
Checks 0 https://github.com/speechbrain/speechbrain/pull/2550/checks
Files changed 21 https://github.com/speechbrain/speechbrain/pull/2550/files
https://github.co/hiddenchars
https://github.com/speechbrain/speechbrain/pull/{{ revealButtonHref }}
https://github.com/BenoitWang
BenoitWanghttps://github.com/BenoitWang
May 16, 2024https://github.com/speechbrain/speechbrain/pull/2550#issue-2300225969
Please reload this pagehttps://github.com/speechbrain/speechbrain/pull/2550
@mravanellihttps://github.com/mravanelli
@poonehmousavihttps://github.com/poonehmousavi
LTU-AShttps://arxiv.org/pdf/2309.14405.pdf
Please reload this pagehttps://github.com/speechbrain/speechbrain/pull/2550
BenoitWanghttps://github.com/BenoitWang
May 16, 2024 12:01https://github.com/speechbrain/speechbrain/pull/2550#commits-pushed-149b3b0
https://github.com/BenoitWang
add ltu-as recipehttps://github.com/speechbrain/speechbrain/pull/2550/commits/149b3b0d0674e6f3c79dad9787599ba0133873ac
149b3b0https://github.com/speechbrain/speechbrain/pull/2550/commits/149b3b0d0674e6f3c79dad9787599ba0133873ac
https://github.com/BenoitWang
enhance LinearWarmupSchedulerhttps://github.com/speechbrain/speechbrain/pull/2550/commits/2a79792285c0598c550117faafe8887d8947b4ca
2a79792https://github.com/speechbrain/speechbrain/pull/2550/commits/2a79792285c0598c550117faafe8887d8947b4ca
https://github.com/BenoitWang
adapt llama2 recipe to the latest modifhttps://github.com/speechbrain/speechbrain/pull/2550/commits/1b6e29579a908d49df92f738e13098cfbceb6184
1b6e295https://github.com/speechbrain/speechbrain/pull/2550/commits/1b6e29579a908d49df92f738e13098cfbceb6184
https://github.com/mravanelli
mravanellihttps://github.com/mravanelli
Jun 17, 2024https://github.com/speechbrain/speechbrain/pull/2550#issuecomment-2173799970
@BenoitWanghttps://github.com/BenoitWang
Please reload this pagehttps://github.com/speechbrain/speechbrain/pull/2550
https://github.com/mravanelli
mravanellihttps://github.com/mravanelli
June 17, 2024 16:24https://github.com/speechbrain/speechbrain/pull/2550#event-13188385348
https://github.com/mravanelli
mravanellihttps://github.com/mravanelli
BenoitWanghttps://github.com/BenoitWang
Jun 17, 2024https://github.com/speechbrain/speechbrain/pull/2550#event-13188386068
https://github.com/mravanelli
mravanellihttps://github.com/mravanelli
enhancement https://github.com/speechbrain/speechbrain/issues?q=state%3Aopen%20label%3Aenhancement
Jun 17, 2024https://github.com/speechbrain/speechbrain/pull/2550#event-13188386772
BenoitWanghttps://github.com/BenoitWang
June 21, 2024 23:52https://github.com/speechbrain/speechbrain/pull/2550#commits-pushed-ec05762
https://github.com/BenoitWang
modify whisper to not pad to 30s if all the input audios are shorthttps://github.com/speechbrain/speechbrain/pull/2550/commits/ec05762bfe8b21d8f451c382a4ce80d0b141f6b5
ec05762https://github.com/speechbrain/speechbrain/pull/2550/commits/ec05762bfe8b21d8f451c382a4ce80d0b141f6b5
https://github.com/BenoitWang
fix linear scheduler examplehttps://github.com/speechbrain/speechbrain/pull/2550/commits/8eacaf332f9198fadb973b188d6e1a0b4264c182
8eacaf3https://github.com/speechbrain/speechbrain/pull/2550/commits/8eacaf332f9198fadb973b188d6e1a0b4264c182
https://github.com/BenoitWang
enhancements, add comments, docstringshttps://github.com/speechbrain/speechbrain/pull/2550/commits/4f598fadf3ab87367919a4c121b325d5d169ea2b
4f598fahttps://github.com/speechbrain/speechbrain/pull/2550/commits/4f598fadf3ab87367919a4c121b325d5d169ea2b
https://github.com/BenoitWang
add an evaluation stagehttps://github.com/speechbrain/speechbrain/pull/2550/commits/8a0220363e382de86cb5ecbd4267dc3d1a559c09
8a02203https://github.com/speechbrain/speechbrain/pull/2550/commits/8a0220363e382de86cb5ecbd4267dc3d1a559c09
https://github.com/BenoitWang
fixeshttps://github.com/speechbrain/speechbrain/pull/2550/commits/51b30f097e174d4bab738295944f75f550f36b27
51b30f0https://github.com/speechbrain/speechbrain/pull/2550/commits/51b30f097e174d4bab738295944f75f550f36b27
https://github.com/BenoitWang
fixeshttps://github.com/speechbrain/speechbrain/pull/2550/commits/13349012947c02d08f4b8d8c46d19e879b7d542b
1334901https://github.com/speechbrain/speechbrain/pull/2550/commits/13349012947c02d08f4b8d8c46d19e879b7d542b
https://github.com/BenoitWang
fixeshttps://github.com/speechbrain/speechbrain/pull/2550/commits/080771b1435c0531692c6a21805e2084f04dc157
080771bhttps://github.com/speechbrain/speechbrain/pull/2550/commits/080771b1435c0531692c6a21805e2084f04dc157
https://github.com/BenoitWang
simplify preparation functionshttps://github.com/speechbrain/speechbrain/pull/2550/commits/91ace43187bd652a7d458909af5ed8fb0fb82d4a
91ace43https://github.com/speechbrain/speechbrain/pull/2550/commits/91ace43187bd652a7d458909af5ed8fb0fb82d4a
https://github.com/BenoitWang
fixeshttps://github.com/speechbrain/speechbrain/pull/2550/commits/b990dd123b63840133eee0e41ed75e4ea10f2735
b990dd1https://github.com/speechbrain/speechbrain/pull/2550/commits/b990dd123b63840133eee0e41ed75e4ea10f2735
https://github.com/BenoitWang
fixeshttps://github.com/speechbrain/speechbrain/pull/2550/commits/db889edc2102b4b21d55088334e91aa662c9d9d0
db889edhttps://github.com/speechbrain/speechbrain/pull/2550/commits/db889edc2102b4b21d55088334e91aa662c9d9d0
https://github.com/BenoitWang
fix yamlshttps://github.com/speechbrain/speechbrain/pull/2550/commits/e1a9e9afb7dc8623938f0f9755ae4cce0ff421b2
e1a9e9ahttps://github.com/speechbrain/speechbrain/pull/2550/commits/e1a9e9afb7dc8623938f0f9755ae4cce0ff421b2
https://github.com/BenoitWang
fix whisper example and preparation feature pathshttps://github.com/speechbrain/speechbrain/pull/2550/commits/185d27d6bba68ea5ea28904521c763d685a5fb7c
185d27dhttps://github.com/speechbrain/speechbrain/pull/2550/commits/185d27d6bba68ea5ea28904521c763d685a5fb7c
https://github.com/BenoitWang
add downloadable linkshttps://github.com/speechbrain/speechbrain/pull/2550/commits/8b863682d33f22c20a2a39e5bcc51844d7b3cc7f
8b86368https://github.com/speechbrain/speechbrain/pull/2550/commits/8b863682d33f22c20a2a39e5bcc51844d7b3cc7f
https://github.com/BenoitWang
add requirements and fix preparationhttps://github.com/speechbrain/speechbrain/pull/2550/commits/e0e09903e9c29c9c772e57076e28b436f8612eba
e0e0990https://github.com/speechbrain/speechbrain/pull/2550/commits/e0e09903e9c29c9c772e57076e28b436f8612eba
https://github.com/BenoitWang
fixhttps://github.com/speechbrain/speechbrain/pull/2550/commits/44ebdc5fbba1a9bbea8c1f126e4bd4ccdc9acc41
44ebdc5https://github.com/speechbrain/speechbrain/pull/2550/commits/44ebdc5fbba1a9bbea8c1f126e4bd4ccdc9acc41
https://github.com/mravanelli
Merge branch 'develop' into speech_llmhttps://github.com/speechbrain/speechbrain/pull/2550/commits/cb0fe66c2eca5e3389f3187c43ae7c85aa26b0da
cb0fe66https://github.com/speechbrain/speechbrain/pull/2550/commits/cb0fe66c2eca5e3389f3187c43ae7c85aa26b0da
https://github.com/BenoitWang
Merge branch 'develop' into speech_llmhttps://github.com/speechbrain/speechbrain/pull/2550/commits/c7baa5880cc088a5cbac9e83c2b9e7b4c1c1dd73
c7baa58https://github.com/speechbrain/speechbrain/pull/2550/commits/c7baa5880cc088a5cbac9e83c2b9e7b4c1c1dd73
https://github.com/BenoitWang
add inference examplehttps://github.com/speechbrain/speechbrain/pull/2550/commits/36150fb764df2e6bd71b081249db5b1a56741019
36150fbhttps://github.com/speechbrain/speechbrain/pull/2550/commits/36150fb764df2e6bd71b081249db5b1a56741019
https://github.com/BenoitWang
Merge branch 'speech_llm' ofhttps://github.com/speechbrain/speechbrain/pull/2550/commits/5cdfc4972ab21d724e8b69759c8eb180c5cc74c2
https://github.com/BenoitWang/speechbrainhttps://github.com/BenoitWang/speechbrain
…https://github.com/speechbrain/speechbrain/pull/2550/commits/5cdfc4972ab21d724e8b69759c8eb180c5cc74c2
5cdfc49https://github.com/speechbrain/speechbrain/pull/2550/commits/5cdfc4972ab21d724e8b69759c8eb180c5cc74c2
https://github.com/BenoitWang
move tltr pretrained weight downloading from stage 0 to stage 1https://github.com/speechbrain/speechbrain/pull/2550/commits/c4d566285e0970553130f69a1623c20cb3ce67a2
c4d5662https://github.com/speechbrain/speechbrain/pull/2550/commits/c4d566285e0970553130f69a1623c20cb3ce67a2
https://github.com/BenoitWang
recipe testhttps://github.com/speechbrain/speechbrain/pull/2550/commits/59a60459e56a9ae1bc61fef0764d8b8e0a414a76
59a6045https://github.com/speechbrain/speechbrain/pull/2550/commits/59a60459e56a9ae1bc61fef0764d8b8e0a414a76
https://github.com/BenoitWang
add all recipe testshttps://github.com/speechbrain/speechbrain/pull/2550/commits/1724cdda73c46c66ba5c71401173f7180f23c676
1724cddhttps://github.com/speechbrain/speechbrain/pull/2550/commits/1724cdda73c46c66ba5c71401173f7180f23c676
https://github.com/BenoitWang
fixhttps://github.com/speechbrain/speechbrain/pull/2550/commits/0173026d58ee33b563e1ce97a4a72a8f67f40236
0173026https://github.com/speechbrain/speechbrain/pull/2550/commits/0173026d58ee33b563e1ce97a4a72a8f67f40236
https://github.com/asumagic
asumagichttps://github.com/asumagic
Sep 10, 2024 https://github.com/speechbrain/speechbrain/pull/2550#ref-issue-2505643636
SpeechLLM and Whisper #2663 https://github.com/speechbrain/speechbrain/issues/2663
Sign up for freehttps://github.com/join?source=comment-repo
Sign in to commenthttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fspeechbrain%2Fspeechbrain%2Fpull%2F2550
https://github.com/mravanelli
mravanelli https://github.com/mravanelli
https://github.com/BenoitWang
BenoitWang https://github.com/BenoitWang
enhancement https://github.com/speechbrain/speechbrain/issues?q=state%3Aopen%20label%3Aenhancement
Please reload this pagehttps://github.com/speechbrain/speechbrain/pull/2550
https://github.com/BenoitWang
https://github.com/mravanelli
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.