{"id":3953,"date":"2024-10-14T16:52:56","date_gmt":"2024-10-14T23:52:56","guid":{"rendered":"http:\/\/blog.light42.com\/wordpress\/?p=3953"},"modified":"2024-10-26T13:25:01","modified_gmt":"2024-10-26T20:25:01","slug":"llms-and-codeagents-by-graham-neubig-cmu","status":"publish","type":"post","link":"http:\/\/blog.light42.com\/wordpress\/?p=3953","title":{"rendered":"LLMs and CodeAgents by Graham Neubig CMU"},"content":{"rendered":"<p>notes from lecture today:<\/p>\n<p>Graham&#8217;s Lab <a href=\"https:\/\/github.com\/orgs\/All-Hands-AI\/repositories\" rel=\"noopener\" target=\"_blank\">LINK<\/a><\/p>\n<p><a href=\"https:\/\/livecodebench.github.io\/\" rel=\"noopener\" target=\"_blank\">LiveCodeBench<\/a>  a comparison framework + <a href=\"https:\/\/livecodebench.github.io\/leaderboard.html\" rel=\"noopener\" target=\"_blank\">leaderboard<\/a> from UC Berkeley<\/p>\n<p><a href=\"https:\/\/gorilla.cs.berkeley.edu\/blogs\/8_berkeley_function_calling_leaderboard.html\" rel=\"noopener\" target=\"_blank\">GorillaCS<\/a>  ability to accurately call a service API, <a href=\"https:\/\/gorilla.cs.berkeley.edu\/leaderboard.html\" rel=\"noopener\" target=\"_blank\">leaderboard<\/a> from UC Berkeley<\/p>\n<p><strong>SWE-Agent<\/strong>  <a href=\"https:\/\/arxiv.org\/abs\/2405.15793\" rel=\"noopener\" target=\"_blank\">paper<\/a><\/p>\n<p><strong>AIDER<\/strong> Repo Understanding using Tree-sitter plus ChatGPT4 services<br \/>\n<a href=\"https:\/\/aider.chat\/docs\/repomap.html\" rel=\"noopener\" target=\"_blank\">docs<\/a>  <a href=\"https:\/\/collaborate.princeton.edu\/en\/publications\/intercode-standardizing-and-benchmarking-interactive-coding-with-\" rel=\"noopener\" target=\"_blank\">ovr<\/a> site<\/p>\n<p><a href=\"https:\/\/tree-sitter.github.io\/tree-sitter\/\" rel=\"noopener\" target=\"_blank\">Tree-sitter<\/a> <a href=\"https:\/\/github.com\/grantjenks\/py-tree-sitter-languages\" rel=\"noopener\" target=\"_blank\">plugins<\/a><\/p>\n<p><strong>Code-Act<\/strong> execute code actions <a href=\"https:\/\/huggingface.co\/datasets\/xingyaoww\/code-act\" rel=\"noopener\" target=\"_blank\">HF<\/a> * <a href=\"https:\/\/github.com\/xingyaoww\/code-act\" rel=\"noopener\" target=\"_blank\">repo<\/a>  <\/p>\n<p><a href=\"https:\/\/arxiv.org\/abs\/2401.14196\" rel=\"noopener\" target=\"_blank\">DeepSeek-Coder<\/a> * <a href=\"https:\/\/arxiv.org\/abs\/2406.11931\" rel=\"noopener\" target=\"_blank\">DeepSeek-Coder v2<\/a>  * <a href=\"https:\/\/arxiv.org\/abs\/2401.02954\">DeepSeek LLM<\/a> <\/p>\n<p><a href=\"https:\/\/arxiv.org\/abs\/2406.01304\" rel=\"noopener\" target=\"_blank\">CodeR<\/a> &#8212; Muti-Agent plus Task Graphs  <\/p>\n<p>OpenCode Interpreter <a href=\"https:\/\/arxiv.org\/abs\/2402.14658\" rel=\"noopener\" target=\"_blank\">paper<\/a>  <a href=\"https:\/\/opencodeinterpreter.github.io\" rel=\"noopener\" target=\"_blank\">site<\/a><\/p>\n<p><em>bonus content<\/em>: <\/p>\n<p> https:\/\/simonwillison.net\/tags\/ai-assisted-programming\/<\/p>\n<p><a href=\"https:\/\/huggingface.co\/Team-ACE\" rel=\"noopener\" target=\"_blank\">Team-ACE<\/a>: a finetuned model of <a href=\"https:\/\/huggingface.co\/meta-llama\/Llama-3.1-8B-Instruct\" rel=\"noopener\" target=\"_blank\">LLaMA-3.1-8B-Instruct<\/a> (<a href=\"https:\/\/build.nvidia.com\/meta\/llama-3_1-8b-instruct\" rel=\"noopener\" target=\"_blank\">demo<\/a> <a href=\"https:\/\/huggingface.co\/Team-ACE\/ToolACE-8B\/tree\/main\" rel=\"noopener\" target=\"_blank\">HF<\/a>)with ZH_Code dataset   <\/p>\n","protected":false},"excerpt":{"rendered":"<p>notes from lecture today: Graham&#8217;s Lab LINK LiveCodeBench a comparison framework + leaderboard from UC Berkeley GorillaCS ability to accurately call a service API, leaderboard from UC Berkeley SWE-Agent paper AIDER Repo Understanding using Tree-sitter plus ChatGPT4 services docs ovr site Tree-sitter plugins Code-Act execute code actions HF * repo DeepSeek-Coder * DeepSeek-Coder v2 * [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"_links":{"self":[{"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/3953"}],"collection":[{"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3953"}],"version-history":[{"count":32,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/3953\/revisions"}],"predecessor-version":[{"id":3986,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/3953\/revisions\/3986"}],"wp:attachment":[{"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3953"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3953"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.light42.com\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3953"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}