René's URL Explorer Experiment

Title: GitHub - modelscope/Trinity-RFT: Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (LLM).

Open Graph Title: GitHub - modelscope/Trinity-RFT: Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (LLM).

X Title: GitHub - modelscope/Trinity-RFT: Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (LLM).

Description: Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (LLM). - modelscope/Trinity-RFT

Open Graph Description: Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (LLM). - modelscope/Trinity-RFT

X Description: Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (LLM). - modelscope/Trinity-RFT

Opengraph URL: https://github.com/modelscope/Trinity-RFT

X: @github

direct link

Domain: github.com

route-pattern	/:user_id/:repository
route-controller	files
route-action	disambiguate
fetch-nonce	v2:5be8159b-be7b-a098-48f3-d1e9faf64235
current-catalog-service-hash	f3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb
request-id	D4FA:3EFE85:1FB5A58:2B2F6FD:6964C09E
html-safe-nonce	62fd296bd4c845adf16b4f08cfbc22a14657575011096621c1a075a165ccca2f
visitor-payload	eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENEZBOjNFRkU4NToxRkI1QTU4OjJCMkY2RkQ6Njk2NEMwOUUiLCJ2aXNpdG9yX2lkIjoiODY5NTI3OTMzNDEzNTM5OTk4IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmac	a5dc4cb0c34da3255067bad0cf830f2a9639d9b5c007401165dc850e6752f0c1
hovercard-subject-tag	repository:963030058
github-keyboard-shortcuts	repository,copilot
google-site-verification	Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-url	https://collector.github.com/github/collect
analytics-location	//
fb:app_id	1401488693436528
apple-itunes-app	app-id=1477376905, app-argument=https://github.com/modelscope/Trinity-RFT
twitter:image	https://opengraph.githubassets.com/a3cf703d609acff434f8a3eb4ccfc156ef3dba78f36001410e9cfa860dbadc22/modelscope/Trinity-RFT
twitter:card	summary_large_image
og:image	https://opengraph.githubassets.com/a3cf703d609acff434f8a3eb4ccfc156ef3dba78f36001410e9cfa860dbadc22/modelscope/Trinity-RFT
og:image:alt	Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (LLM). - modelscope/Trinity-RFT
og:image:width	1200
og:image:height	600
og:site_name	GitHub
og:type	object
hostname	github.com
expected-hostname	github.com
None	21df671ce2c9f1a16940ccbd3af6cb4f3f12a856929ca7eb1b4aea8e384ea442
turbo-cache-control	no-preview
go-import	github.com/modelscope/Trinity-RFT git https://github.com/modelscope/Trinity-RFT.git
octolytics-dimension-user_id	109945100
octolytics-dimension-user_login	modelscope
octolytics-dimension-repository_id	963030058
octolytics-dimension-repository_nwo	modelscope/Trinity-RFT
octolytics-dimension-repository_public	true
octolytics-dimension-repository_is_fork	false
octolytics-dimension-repository_network_root_id	963030058
octolytics-dimension-repository_network_root_nwo	modelscope/Trinity-RFT
turbo-body-classes	logged-out env-production page-responsive
disable-turbo	false
browser-stats-url	https://api.github.com/_private/browser/stats
browser-errors-url	https://api.github.com/_private/browser/errors
release	5707c685ac172d50a0bdd7533dde4f8aabcf8eef
ui-target	full
theme-color	#1e2327
color-scheme	light dark

Links:

Skip to content	https://github.com/modelscope/Trinity-RFT#start-of-content
	https://github.com/
Sign in	https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fmodelscope%2FTrinity-RFT
GitHub CopilotWrite better code with AI	https://github.com/features/copilot
GitHub SparkBuild and deploy intelligent apps	https://github.com/features/spark
GitHub ModelsManage and compare prompts	https://github.com/features/models
MCP RegistryNewIntegrate external tools	https://github.com/mcp
ActionsAutomate any workflow	https://github.com/features/actions
CodespacesInstant dev environments	https://github.com/features/codespaces
IssuesPlan and track work	https://github.com/features/issues
Code ReviewManage code changes	https://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilities	https://github.com/security/advanced-security
Code securitySecure your code as you build	https://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they start	https://github.com/security/advanced-security/secret-protection
Why GitHub	https://github.com/why-github
Documentation	https://docs.github.com
Blog	https://github.blog
Changelog	https://github.blog/changelog
Marketplace	https://github.com/marketplace
View all features	https://github.com/features
Enterprises	https://github.com/enterprise
Small and medium teams	https://github.com/team
Startups	https://github.com/enterprise/startups
Nonprofits	https://github.com/solutions/industry/nonprofits
App Modernization	https://github.com/solutions/use-case/app-modernization
DevSecOps	https://github.com/solutions/use-case/devsecops
DevOps	https://github.com/solutions/use-case/devops
CI/CD	https://github.com/solutions/use-case/ci-cd
View all use cases	https://github.com/solutions/use-case
Healthcare	https://github.com/solutions/industry/healthcare
Financial services	https://github.com/solutions/industry/financial-services
Manufacturing	https://github.com/solutions/industry/manufacturing
Government	https://github.com/solutions/industry/government
View all industries	https://github.com/solutions/industry
View all solutions	https://github.com/solutions
AI	https://github.com/resources/articles?topic=ai
Software Development	https://github.com/resources/articles?topic=software-development
DevOps	https://github.com/resources/articles?topic=devops
Security	https://github.com/resources/articles?topic=security
View all topics	https://github.com/resources/articles
Customer stories	https://github.com/customer-stories
Events & webinars	https://github.com/resources/events
Ebooks & reports	https://github.com/resources/whitepapers
Business insights	https://github.com/solutions/executive-insights
GitHub Skills	https://skills.github.com
Documentation	https://docs.github.com
Customer support	https://support.github.com
Community forum	https://github.com/orgs/community/discussions
Trust center	https://github.com/trust-center
Partners	https://github.com/partners
GitHub SponsorsFund open source developers	https://github.com/sponsors
Security Lab	https://securitylab.github.com
Maintainer Community	https://maintainers.github.com
Accelerator	https://github.com/accelerator
Archive Program	https://archiveprogram.github.com
Topics	https://github.com/topics
Trending	https://github.com/trending
Collections	https://github.com/collections
Enterprise platformAI-powered developer platform	https://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security features	https://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI features	https://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 support	https://github.com/premium-support
Pricing	https://github.com/pricing
Search syntax tips	https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentation	https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in	https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fmodelscope%2FTrinity-RFT
Sign up	https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E&source=header-repo&source_repo=modelscope%2FTrinity-RFT
Reload	https://github.com/modelscope/Trinity-RFT
Reload	https://github.com/modelscope/Trinity-RFT
Reload	https://github.com/modelscope/Trinity-RFT
modelscope	https://github.com/modelscope
Trinity-RFT	https://github.com/modelscope/Trinity-RFT
Notifications	https://github.com/login?return_to=%2Fmodelscope%2FTrinity-RFT
Fork 47	https://github.com/login?return_to=%2Fmodelscope%2FTrinity-RFT
Star 474	https://github.com/login?return_to=%2Fmodelscope%2FTrinity-RFT
modelscope.github.io/Trinity-RFT/	https://modelscope.github.io/Trinity-RFT/
Apache-2.0 license	https://github.com/modelscope/Trinity-RFT/blob/main/LICENSE
474 stars	https://github.com/modelscope/Trinity-RFT/stargazers
47 forks	https://github.com/modelscope/Trinity-RFT/forks
Branches	https://github.com/modelscope/Trinity-RFT/branches
Tags	https://github.com/modelscope/Trinity-RFT/tags
Activity	https://github.com/modelscope/Trinity-RFT/activity
Star	https://github.com/login?return_to=%2Fmodelscope%2FTrinity-RFT
Notifications	https://github.com/login?return_to=%2Fmodelscope%2FTrinity-RFT
Code	https://github.com/modelscope/Trinity-RFT
Issues 29	https://github.com/modelscope/Trinity-RFT/issues
Pull requests 4	https://github.com/modelscope/Trinity-RFT/pulls
Discussions	https://github.com/modelscope/Trinity-RFT/discussions
Actions	https://github.com/modelscope/Trinity-RFT/actions
Projects 0	https://github.com/modelscope/Trinity-RFT/projects
Security Uh oh! There was an error while loading. Please reload this page.	https://github.com/modelscope/Trinity-RFT/security
Please reload this page	https://github.com/modelscope/Trinity-RFT
Insights	https://github.com/modelscope/Trinity-RFT/pulse
Code	https://github.com/modelscope/Trinity-RFT
Issues	https://github.com/modelscope/Trinity-RFT/issues
Pull requests	https://github.com/modelscope/Trinity-RFT/pulls
Discussions	https://github.com/modelscope/Trinity-RFT/discussions
Actions	https://github.com/modelscope/Trinity-RFT/actions
Projects	https://github.com/modelscope/Trinity-RFT/projects
Security	https://github.com/modelscope/Trinity-RFT/security
Insights	https://github.com/modelscope/Trinity-RFT/pulse
Branches	https://github.com/modelscope/Trinity-RFT/branches
Tags	https://github.com/modelscope/Trinity-RFT/tags
	https://github.com/modelscope/Trinity-RFT/branches
	https://github.com/modelscope/Trinity-RFT/tags
380 Commits	https://github.com/modelscope/Trinity-RFT/commits/main/
	https://github.com/modelscope/Trinity-RFT/commits/main/
.github	https://github.com/modelscope/Trinity-RFT/tree/main/.github
.github	https://github.com/modelscope/Trinity-RFT/tree/main/.github
benchmark	https://github.com/modelscope/Trinity-RFT/tree/main/benchmark
benchmark	https://github.com/modelscope/Trinity-RFT/tree/main/benchmark
docs	https://github.com/modelscope/Trinity-RFT/tree/main/docs
docs	https://github.com/modelscope/Trinity-RFT/tree/main/docs
environments	https://github.com/modelscope/Trinity-RFT/tree/main/environments
environments	https://github.com/modelscope/Trinity-RFT/tree/main/environments
examples	https://github.com/modelscope/Trinity-RFT/tree/main/examples
examples	https://github.com/modelscope/Trinity-RFT/tree/main/examples
scripts	https://github.com/modelscope/Trinity-RFT/tree/main/scripts
scripts	https://github.com/modelscope/Trinity-RFT/tree/main/scripts
tests	https://github.com/modelscope/Trinity-RFT/tree/main/tests
tests	https://github.com/modelscope/Trinity-RFT/tree/main/tests
trinity	https://github.com/modelscope/Trinity-RFT/tree/main/trinity
trinity	https://github.com/modelscope/Trinity-RFT/tree/main/trinity
.flake8	https://github.com/modelscope/Trinity-RFT/blob/main/.flake8
.flake8	https://github.com/modelscope/Trinity-RFT/blob/main/.flake8
.gitignore	https://github.com/modelscope/Trinity-RFT/blob/main/.gitignore
.gitignore	https://github.com/modelscope/Trinity-RFT/blob/main/.gitignore
.pre-commit-config.yaml	https://github.com/modelscope/Trinity-RFT/blob/main/.pre-commit-config.yaml
.pre-commit-config.yaml	https://github.com/modelscope/Trinity-RFT/blob/main/.pre-commit-config.yaml
CONTRIBUTING.md	https://github.com/modelscope/Trinity-RFT/blob/main/CONTRIBUTING.md
CONTRIBUTING.md	https://github.com/modelscope/Trinity-RFT/blob/main/CONTRIBUTING.md
LICENSE	https://github.com/modelscope/Trinity-RFT/blob/main/LICENSE
LICENSE	https://github.com/modelscope/Trinity-RFT/blob/main/LICENSE
README.md	https://github.com/modelscope/Trinity-RFT/blob/main/README.md
README.md	https://github.com/modelscope/Trinity-RFT/blob/main/README.md
README_zh.md	https://github.com/modelscope/Trinity-RFT/blob/main/README_zh.md
README_zh.md	https://github.com/modelscope/Trinity-RFT/blob/main/README_zh.md
pyproject.toml	https://github.com/modelscope/Trinity-RFT/blob/main/pyproject.toml
pyproject.toml	https://github.com/modelscope/Trinity-RFT/blob/main/pyproject.toml
setup.py	https://github.com/modelscope/Trinity-RFT/blob/main/setup.py
setup.py	https://github.com/modelscope/Trinity-RFT/blob/main/setup.py
README	https://github.com/modelscope/Trinity-RFT
Contributing	https://github.com/modelscope/Trinity-RFT
Apache-2.0 license	https://github.com/modelscope/Trinity-RFT
中文主页	https://github.com/modelscope/Trinity-RFT/blob/main/README_zh.md
Tutorial	https://modelscope.github.io/Trinity-RFT/
FAQ	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/faq.html
	https://camo.githubusercontent.com/3332441bbfcdb42e2b842c96da1c0771a54511b7a52a946ca5a21e4be74bd4b7/68747470733a2f2f696d672e616c6963646e2e636f6d2f696d6765787472612f69312f4f31434e30316c764c7066773235506c346f68475a6e555f2121363030303030303030373531392d322d7470732d313632382d3439302e706e67
	https://github.com/modelscope/Trinity-RFT#trinity-rft-a-general-purpose-and-unified-framework-forreinforcement-fine-tuning-of-large-language-models
	https://arxiv.org/abs/2505.17826
	https://modelscope.github.io/Trinity-RFT/
	https://pypi.org/project/trinity-rft/
	https://camo.githubusercontent.com/cbbee0a45a727ab338a068488b57af8dca9ab5be7046469577902c5257107fc3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4170616368652d2d322e302d3030303030302e737667
	https://github.com/modelscope/Trinity-RFT#-what-is-trinity-rft
[tutorial]	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/develop_workflow.html
[tutorial]	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/develop_algorithm.html
[tutorial]	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/develop_operator.html
	https://github.com/modelscope/Trinity-RFT#-news
[Release Notes]	https://github.com/modelscope/Trinity-RFT/releases/tag/v0.4.0
Tinker	https://thinkingmachines.ai/tinker/
News	https://tech.china.com.cn/sx/20251201/411376.shtml
Release Notes	https://github.com/modelscope/Trinity-RFT/releases/tag/v0.3.3
Learn-to-Ask	https://github.com/modelscope/Trinity-RFT/tree/main/examples/learn_to_ask
paper	https://arxiv.org/pdf/2510.25441
BOTS	https://github.com/modelscope/Trinity-RFT/tree/main/examples/bots
paper	https://arxiv.org/pdf/2510.26374
Our paper	https://arxiv.org/pdf/2509.24203
implementation	https://github.com/modelscope/Trinity-RFT/tree/main/examples/rec_gsm8k
CHORD	https://github.com/modelscope/Trinity-RFT/tree/main/examples/mix_chord
paper	https://arxiv.org/pdf/2508.11408
https://arxiv.org/abs/2505.17826	https://arxiv.org/abs/2505.17826
https://arxiv.org/abs/2505.17826	https://arxiv.org/abs/2505.17826
	https://github.com/modelscope/Trinity-RFT#-tutorials-and-guidelines
Quick start: GRPO on GSM8k	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_reasoning_basic.html
Off-policy RFT	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_reasoning_advanced.html
Fully asynchronous RFT	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_async_mode.html
Offline learning by DPO or SFT	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_dpo.html
RFT without local GPU (Tinker Backend)	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_tinker_backend.html
Concatenated multi-turn workflow	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_multi_turn.html
General multi-step workflow	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_step_wise.html
ReAct workflow with an agent framework	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_react.html
Example: train a web-search agent	https://github.com/modelscope/Trinity-RFT/tree/main/examples/agentscope_websearch
Rollout task mixing and selection	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/develop_selector.html
Online task curriculum	https://github.com/modelscope/Trinity-RFT/tree/main/examples/bots
paper	https://arxiv.org/pdf/2510.26374
Research project: learn-to-ask	https://github.com/modelscope/Trinity-RFT/tree/main/examples/learn_to_ask
paper	https://arxiv.org/pdf/2510.25441
Experience replay with prioritization	https://github.com/modelscope/Trinity-RFT/tree/main/examples/ppo_countdown_exp_replay
Advanced data processing & human-in-the-loop	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_data_functionalities.html
RL algorithm development with Trinity-RFT	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_mix_algo.html
paper	https://arxiv.org/pdf/2508.11408
Research project: group-relative REINFORCE	https://github.com/modelscope/Trinity-RFT/tree/main/examples/rec_gsm8k
paper	https://arxiv.org/abs/2509.24203
RULER	https://github.com/modelscope/Trinity-RFT/tree/main/examples/grpo_gsm8k_ruler
trainable RULER	https://github.com/modelscope/Trinity-RFT/tree/main/examples/grpo_gsm8k_trainable_ruler
rubric-as-reward	https://github.com/modelscope/Trinity-RFT/tree/main/examples/grpo_rubric_as_reward
Benchmark toolkit (quick verification & experimentation)	https://github.com/modelscope/Trinity-RFT/tree/main/benchmark/README.md
Guru-Math benchmark & comparison with veRL	https://github.com/modelscope/Trinity-RFT/tree/main/benchmark/reports/guru_math.md
FrozenLake benchmark & comparison with rLLM	https://github.com/modelscope/Trinity-RFT/tree/main/benchmark/reports/frozenlake.md
Alfworld benchmark & comparison with rLLM	https://github.com/modelscope/Trinity-RFT/tree/main/benchmark/reports/alfworld.md
Full configurations	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/trinity_configs.html
GPU resource and training configuration guide	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/trinity_gpu_configs.html
Understand the coordination between explorer and trainer	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/synchronizer.html
How to align configuration with veRL	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/align_with_verl.html
Trinity-RFT documentation	https://modelscope.github.io/Trinity-RFT/
	https://github.com/modelscope/Trinity-RFT#-key-features
	https://camo.githubusercontent.com/5df82684977ebca5e2f724e009ac6821caf617fc5a8b946013b8e5fca79bfa37/68747470733a2f2f696d672e616c6963646e2e636f6d2f696d6765787472612f69332f4f31434e303145374e736b533146466f5449396a6c61515f2121363030303030303030303435382d322d7470732d313435382d3638322e706e67
AgentScope	https://github.com/agentscope-ai/agentscope
	https://camo.githubusercontent.com/efa9a0947bda535179acb6e6cbb4c353cba445b4079459fcc486746e2896bf9d/68747470733a2f2f696d672e616c6963646e2e636f6d2f696d6765787472612f69312f4f31434e30317a3169376b6b316a6c4d455661385a48565f2121363030303030303030343538382d322d7470732d313236322d3639352e706e67
	https://camo.githubusercontent.com/12dc5a3efd173a1a5bce3131faca075720e708766f31d8a913dcb2e62e17556e/68747470733a2f2f696d672e616c6963646e2e636f6d2f696d6765787472612f69322f4f31434e3031476b3943527732384e734c30396e624f6a5f2121363030303030303030373932312d322d7470732d323533302d3636302e706e67
	https://camo.githubusercontent.com/1f2782a7a9c95fbdec4da39720c12ca02a0a6ab258c58ca8fb56aa7f533b0e08/68747470733a2f2f696d672e616c6963646e2e636f6d2f696d6765787472612f69312f4f31434e30315469306f343332305279776f417579684e5f2121363030303030303030363834372d322d7470732d333834302d323133342e706e67
	https://github.com/modelscope/Trinity-RFT#-supported-algorithms
Algorithm module	https://github.com/modelscope/Trinity-RFT/blob/main/trinity/algorithm/algorithm.py
tutorial	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/develop_algorithm.html
Paper	https://arxiv.org/pdf/1707.06347
Doc	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_reasoning_basic.html
Countdown Example	https://github.com/modelscope/Trinity-RFT/tree/main/examples/ppo_countdown
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/policy_loss_fn/ppo_policy_loss.py
Paper	https://arxiv.org/pdf/2402.03300
Doc	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_reasoning_basic.html
GSM8K Example	https://github.com/modelscope/Trinity-RFT/tree/main/examples/grpo_gsm8k
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/advantage_fn/grpo_advantage.py
Paper	https://arxiv.org/pdf/2508.11408
Doc	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_mix_algo.html
ToolACE Example	https://github.com/modelscope/Trinity-RFT/blob/main/examples/mix_chord/mix_chord_toolace.yaml
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/policy_loss_fn/chord_policy_loss.py
Paper	https://arxiv.org/pdf/2509.24203
GSM8K Example	https://github.com/modelscope/Trinity-RFT/tree/main/examples/rec_gsm8k
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/policy_loss_fn/rec_policy_loss.py
Paper	https://arxiv.org/pdf/2402.14740
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/advantage_fn/rloo_advantage.py
Paper	https://arxiv.org/pdf/2501.03262
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/advantage_fn/reinforce_advantage.py
Paper	https://arxiv.org/pdf/2507.18071
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/policy_loss_fn/gspo_policy_loss.py
Paper	https://arxiv.org/pdf/2503.14286
GSM8K Example	https://github.com/modelscope/Trinity-RFT/tree/main/examples/topr_gsm8k
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/policy_loss_fn/topr_policy_loss.py
Paper	https://arxiv.org/pdf/2108.05828
GSM8K Example	https://github.com/modelscope/Trinity-RFT/tree/main/examples/sppo_gsm8k
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/policy_loss_fn/sppo_loss_fn.py
Paper	https://arxiv.org/pdf/2506.20520
GSM8K Example	https://github.com/modelscope/Trinity-RFT/tree/main/examples/asymre_gsm8k
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/advantage_fn/asymre_advantage.py
Paper	https://arxiv.org/pdf/2506.13585
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/policy_loss_fn/cispo_policy_loss.py
Paper	https://arxiv.org/pdf/2511.20347
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/algorithm/policy_loss_fn/sapo_policy_loss.py
Blog	https://thinkingmachines.ai/blog/on-policy-distillation/
Paper	https://arxiv.org/pdf/2306.13649
GSM8K Example	https://github.com/modelscope/Trinity-RFT/tree/main/examples/on_policy_distill
Code	https://github.com/modelscope/Trinity-RFT/tree/main/trinity/common/workflows/on_policy_distill_workflow.py
	https://github.com/modelscope/Trinity-RFT#table-of-contents
Quick Start	https://github.com/modelscope/Trinity-RFT#quick-start
Step 1: installation	https://github.com/modelscope/Trinity-RFT#step-1-installation
Step 2: prepare dataset and model	https://github.com/modelscope/Trinity-RFT#step-2-prepare-dataset-and-model
Step 3: configurations	https://github.com/modelscope/Trinity-RFT#step-3-configurations
Step 4: run the RFT process	https://github.com/modelscope/Trinity-RFT#step-4-run-the-rft-process
Contribution Guide	https://github.com/modelscope/Trinity-RFT#contribution-guide
Acknowledgements	https://github.com/modelscope/Trinity-RFT#acknowledgements
Citation	https://github.com/modelscope/Trinity-RFT#citation
	https://github.com/modelscope/Trinity-RFT#quick-start
Tinker training example	https://github.com/modelscope/Trinity-RFT/tree/main/examples/tinker
	https://github.com/modelscope/Trinity-RFT#step-1-installation
	https://github.com/modelscope/Trinity-RFT#from-source-recommended
	https://github.com/modelscope/Trinity-RFT#1-clone-the-repository
	https://github.com/modelscope/Trinity-RFT#2-set-up-environment
	https://github.com/modelscope/Trinity-RFT#using-pre-built-docker-image-recommended-for-beginners
	https://github.com/modelscope/Trinity-RFT#using-conda
	https://github.com/modelscope/Trinity-RFT#using-venv
	https://github.com/modelscope/Trinity-RFT#using-uv
uv	https://github.com/astral-sh/uv
	https://github.com/modelscope/Trinity-RFT#via-pypi
Megatron-LM Backend	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/example_megatron.html
	https://github.com/modelscope/Trinity-RFT#step-2-prepare-dataset-and-model
Huggingface	https://huggingface.co/docs/huggingface_hub/main/en/guides/cli
ModelScope	https://modelscope.cn/docs/models/download
Huggingface	https://huggingface.co/docs/huggingface_hub/main/en/guides/cli#download-a-dataset-or-a-space
ModelScope	https://modelscope.cn/docs/datasets/download
	https://github.com/modelscope/Trinity-RFT#step-3-configurations
examples	https://github.com/modelscope/Trinity-RFT/blob/main/examples
Trinity-Studio	https://github.com/modelscope/Trinity-Studio
	https://camo.githubusercontent.com/292a5bcfdc3f75de3cadfd84549a010b912aed5cd6df1d3134d46153f0f102b4/68747470733a2f2f696d672e616c6963646e2e636f6d2f696d6765787472612f69312f4f31434e3031796859725630316c474b636874797753485f2121363030303030303030343739312d322d7470732d313438302d3834342e706e67
	https://github.com/modelscope/Trinity-RFT#step-4-run-the-rft-process
Wandb	https://docs.wandb.ai/quickstart/
TensorBoard	https://www.tensorflow.org/tensorboard
MLFlow	https://mlflow.org
this documentation	https://modelscope.github.io/Trinity-RFT/en/main/tutorial/trinity_configs.html#monitor-configuration
	https://github.com/modelscope/Trinity-RFT#contribution-guide
CONTRIBUTING.md	https://github.com/modelscope/Trinity-RFT/blob/main/CONTRIBUTING.md
	https://github.com/modelscope/Trinity-RFT#acknowledgements
verl	https://github.com/volcengine/verl
FSDP	https://pytorch.org/docs/stable/fsdp.html
Megatron-LM	https://github.com/NVIDIA/Megatron-LM
vLLM	https://github.com/vllm-project/vllm
Data-Juicer	https://github.com/modelscope/data-juicer?tab=readme-ov-file
AgentScope	https://github.com/agentscope-ai/agentscope
Ray	https://github.com/ray-project/ray
OpenRLHF	https://github.com/OpenRLHF/OpenRLHF
TRL	https://github.com/huggingface/trl
ChatLearn	https://github.com/alibaba/ChatLearn
rLLM	https://github.com/rllm-org/rllm
	https://github.com/modelscope/Trinity-RFT#citation
modelscope.github.io/Trinity-RFT/	https://modelscope.github.io/Trinity-RFT/
agent	https://github.com/topics/agent
llm	https://github.com/topics/llm
rlhf	https://github.com/topics/rlhf
Readme	https://github.com/modelscope/Trinity-RFT#readme-ov-file
Apache-2.0 license	https://github.com/modelscope/Trinity-RFT#Apache-2.0-1-ov-file
Contributing	https://github.com/modelscope/Trinity-RFT#contributing-ov-file
Please reload this page	https://github.com/modelscope/Trinity-RFT
Activity	https://github.com/modelscope/Trinity-RFT/activity
Custom properties	https://github.com/modelscope/Trinity-RFT/custom-properties
474 stars	https://github.com/modelscope/Trinity-RFT/stargazers
5 watching	https://github.com/modelscope/Trinity-RFT/watchers
47 forks	https://github.com/modelscope/Trinity-RFT/forks
Report repository	https://github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fmodelscope%2FTrinity-RFT&report=modelscope+%28user%29
Releases 9	https://github.com/modelscope/Trinity-RFT/releases
v0.4.0 Latest Dec 30, 2025	https://github.com/modelscope/Trinity-RFT/releases/tag/v0.4.0
+ 8 releases	https://github.com/modelscope/Trinity-RFT/releases
Packages 0	https://github.com/orgs/modelscope/packages?repo_name=Trinity-RFT
Please reload this page	https://github.com/modelscope/Trinity-RFT
Please reload this page	https://github.com/modelscope/Trinity-RFT
Contributors 20	https://github.com/modelscope/Trinity-RFT/graphs/contributors
	https://github.com/pan-x-c
	https://github.com/hiyuchang
	https://github.com/chenyushuo
	https://github.com/yanxi-chen
	https://github.com/garyzhang99
	https://github.com/HYLcool
	https://github.com/shiweijiezero
	https://github.com/yaochaorui
	https://github.com/yxdyc
	https://github.com/binary-husky
	https://github.com/apps/gemini-code-assist
	https://github.com/vadimkantorov
	https://github.com/nkkarpov
	https://github.com/0x404
+ 6 contributors	https://github.com/modelscope/Trinity-RFT/graphs/contributors
Python 99.6%	https://github.com/modelscope/Trinity-RFT/search?l=python
	https://github.com
Terms	https://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacy	https://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Security	https://github.com/security
Status	https://www.githubstatus.com/
Community	https://github.community/
Docs	https://docs.github.com/
Contact	https://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.