
Google Colab ã§ LangChain + RWKV ã詊ã
ãGoogle Colabãã§ãLangChain + RWKVãã詊ããã®ã§ããŸãšããŸããã
ãæ³šæããGoogle Colabãã§ãRWKV-v4-14Bããå®è¡ããã«ã¯ããGoogle Colab Pro/Pro+ãã®ãã¬ãã¢ã ãå¿ èŠã§ãã
åå
1. LangChain + RWKV
ãLangChainãã¯ãGPT-4ããGPT-3.5ãã ãã§ãªããæ§ã ãªèšèªã¢ãã«ãLLMãšããŠå©çšã§ããŸããããããGPT-4ããGPT-3.5ã以å€ã§ããLangChainãã®ãšãŒãžã§ã³ãæ©èœãå¶åŸ¡ã§ããã»ã©ã®èœåãæã€ãLLMãã¯ããŸã ãããŸããã
ããã§ä»åã¯ããLangChain + RWKVãã䜿ã£ãŠããLlamaIndexãã®QAããã³ãããã©ã®çšåºŠå¶åŸ¡ã§ãããã確èªããããšã«ããŸãã
ã»gpt-index/prompts/default_prompts.py
DEFAULT_TEXT_QA_PROMPT_TMPL = (
"Context information is below. \n"
"---------------------\n"
"{context_str}"
"\n---------------------\n"
"Given the context information and not prior knowledge, "
"answer the question: {query_str}\n"
)
DEFAULT_TEXT_QA_PROMPT = QuestionAnswerPrompt(DEFAULT_TEXT_QA_PROMPT_TMPL)2. Colabã§ã®å®è¡
Colabã§ã®å®è¡æé ã¯ã次ã®ãšããã§ãã
(1) ã¡ãã¥ãŒãç·šéâããŒãããã¯ã®èšå®ãã§ããããŒããŠã§ã¢ã¢ã¯ã»ã©ã¬ãŒã¿ãã§ãGPUãã®ããã¬ãã¢ã ããéžæã
(2) Googleãã©ã€ãã®ããŠã³ãã
# Googleãã©ã€ãã®ããŠã³ã
from google.colab import drive
drive.mount("/content/drive")(3) äœæ¥ãã©ã«ããžã®ç§»åã
# äœæ¥ãã©ã«ããžã®ç§»å
import os
os.makedirs("/content/drive/My Drive/work", exist_ok=True)
%cd "/content/drive/My Drive/work"(4) ã¢ãã«ãããŠã³ããŒãããŠãäœæ¥ãã©ã«ã (work) ã«é
眮ã
ä»åã¯ããRWKV-4-Raven-14B-v9-Eng99%-Other1%-20230412-ctx8192.pthãã䜿ããŸããã
(5) ããŒã¯ã³ãã¡ã€ã«ãããŠã³ããŒãããŠãäœæ¥ãã©ã«ã (work) ã«é 眮ã
(6) ããã±ãŒãžã®ã€ã³ã¹ããŒã«ã
# ããã±ãŒãžã®ã€ã³ã¹ããŒã«
!pip install langchain
!pip install rwkv
!pip install tokenizers(7) LLMã®æºåã
%%time
# LLMã®æºå
llm = RWKV(
model="./RWKV-4-Raven-14B-v9-Eng99%-Other1%-20230412-ctx8192.pth",
strategy="cuda fp16",
tokens_path="./20B_tokenizer.json"
)Wall time: 3min 56s(9) Instructããã³ããã®çæé¢æ°ã®æºåã
RWKVãåŠç¿æã«å©çšããŠããAlpacaæžåŒãçæãã颿°ã«ãªããŸãã
from langchain.llms import RWKV
# Instructããã³ããã®çæ
def generate_prompt(instruction, input=None):
if input:
return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
# Instruction:
{instruction}
# Input:
{input}
# Response:
"""
else:
return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.
# Instruction:
{instruction}
# Response:
"""(10) æšè«ã®å®è¡ã
print(llm(generate_prompt("æ¥æ¬ã®éŠéœã¯ïŒ")))æ¥æ¬ã®éŠéœã¯æ±äº¬ã§ãã(11) æšè«ã®å®è¡ã
print(llm(generate_prompt("æ¥æ¬ã§äžçªäººæ°ã®ã¢ãã¡ã¯ïŒ")))æ¥æ¬ã§äžçªäººæ°ã®ã¢ãã¡ã¯ãã忢åµã³ãã³ãã§ãããã®ã¢ãã¡ã¯äžççã«ãé«ãè©äŸ¡ãããŠããããããã·ãªãŒãºãšãªããŸãããã¢ãã¡ã§ã¯ã䞻人å
¬ãã¡ãããã«ããŠååããããçžæãåãããæããŠããŸãã
ãŸããã鲿ã®å·šäººãããèŒãéŒã®ã¢ã«ããžãªããªã©ã人æ°ãéããŠããŸãã3. LlamaIndexã®åºæ¬ããã³ããã®åäœç¢ºèª
LlamaIndexã®QAããã³ããã®åäœã確èªããŸããRWKVã¯ã質åå¿çãAlpacaã®Instructããã³ããã§åŠç¿ããŠããã®ã§ããã®æžåŒã詊ããŠã¿ãŸãã
3-1. æ¥æ¬èª + LlamaIndexã®QAããã³ãã
質åå 容ãçè§£ã§ããŠãªãããã§ãã
qa_prompt="""Context information is below.
---------------------
ã®ã¿ãŒããŒããŒ
ã²ãšããåç»æçš¿ãµã€ãã§æŽ»åããéã«çšãããã³ãã«ããŒã ãã²ãšããæ®æ®µäººåã«åºããªããå
é¢ã«æ±ããŠããæ¿èªæ¬²æ±ã®è±¡åŸŽãšãèšãããã®ã§ãçŸå®ã®ã²ãšãããã¯ããé¢ããããã¹ã±éšãšãŒã¹ã®åœŒæ°æã¡ãããã©ã€ã³ã®åéæ°ã¯1000人è¶
ãããšãã£ãèšå®ãã€ããããŠãã[泚 11][38]ãé¡ãé ãã¢ã³ã°ã«ããæ®åœ±ããã人æ°ãã³ãã®ã«ããŒåç»ãæçš¿ããŠãããæçš¿ãµã€ãã§ã®ç»é²è
æ°ã¯8äžäººè¿ã[39]ããœãããâ¡ãã¿ã®ãããªé³æ¥œé¢ä¿è
ãããæ³šç®ãéããŠãããåç»ã¯çŽæš¹ã«ãã£ãŠåçåãããŠãããæåç¥ã©ã€ãã§å£ããã®ã¿ãŒãæ°èª¿ããè²»çšçã«å
ãŠããã[40]ã
ã©ã€ãã§ã®çŸåšã®èªèº«ã®å®åã§ã¯ã®ã¿ãŒããŒããŒã®ååã§ãã¡ã³ãéããŠãæºè¶³ãããã ãã®æŒå¥ãã§ããªããšããã²ãšãã®èããããæ£äœããçµæãã³ãã®åŸè€ã²ãšããã§ããããšã¯å
¬ã«ã¯ãããŠããªã[泚 12]ã
---------------------
Given the context information and not prior knowledge, answer the question: ã²ãšãã¡ããã®åŸæãªæ¥œåšã¯ïŒ"""
print(llm(qa_prompt))Answer in English.
Answer: I'm sorry, but I cannot answer this question as it is incomplete and requires more context information to provide a meaningful answer. Please provide more details or context for me to generate a meaningful response
ã翻蚳ã
è±èªã§çããã
åç: ç³ãèš³ãããŸãããããã®è³ªåã¯äžå®å šã§ãããæå³ã®ããåçãæäŸããã«ã¯ããå€ãã®ã³ã³ããã¹ãæ å ±ãå¿ èŠã§ããããããçãã§ããŸããã æå³ã®ããåçãåŸãããã«ã詳现ãŸãã¯ã³ã³ããã¹ããæäŸããŠãã ãã
3-2. è±èª + LlamaIndexã®QAããã³ãã
å°ã質åå 容ãçè§£ããŠããã§ããããŸã ãŸã ã§ãã
qa_prompt="""Context information is below.
---------------------
guitar hero
A handle name used by one person when working on a video posting site. It can be said that it is a symbol of the desire for approval that one person can not usually show in public, but is far from being a real person. is attached [Note 11][38]. He posts cover videos of popular bands shot from angles that hide his face, and the number of registrants on the posting site is nearly 80,000 [39]. He has also attracted attention from music insiders like Poison Yami. The video was monetized by Naoki, and was used for the cost of replacing the broken guitar at the school festival live [40].
With his current ability in live performances, he thinks that even if he gathers fans under the name of Guitar Hero, he cannot perform enough to satisfy them. No [Note 12].
---------------------
Given the context information and not prior knowledge, answer the question: Hitori-chan's favorite instrument?"""
print(llm(qa_prompt))Answer: It is not possible to determine the exact favorite instrument of Hitori-chan based on the given context information. The context information provided only mentions that Hitori-chan has a guitar, and that he sometimes uses it for live performances. However, it does not provide any information about his preferred genre or style of music, or whether he has a particular instrument that he prefers to use for live performances. Additionally, the context information does not mention any other details about Hitori-chan's personal life or interests that could help to identify his favorite instrument. Therefore, it is not possible to determine Hitori-chan's favorite instrument based on the given context information alone.
ã翻蚳ã
åç: äžããããã³ã³ããã¹ãæ å ±ã«åºã¥ããŠãã²ãšãã¡ããã®æ£ç¢ºãªãæ°ã«å ¥ãã®æ¥œåšãç¹å®ããããšã¯ã§ããŸããã æäŸãããã³ã³ããã¹ãæ å ±ã«ã¯ãã²ãšãã¡ãããã®ã¿ãŒãæã£ãŠããŠãã©ã€ã ããã©ãŒãã³ã¹ã§æã ã®ã¿ãŒã䜿çšããŠããããšã ããèšåãããŠããŸãã ãã ãã圌ã奜ã鳿¥œã®ãžã£ã³ã«ãã¹ã¿ã€ã«ããŸãã¯ã©ã€ãããã©ãŒãã³ã¹ã«äœ¿çšããããšã奜ãç¹å®ã®æ¥œåšãæã£ãŠãããã©ããã«ã€ããŠã®æ å ±ã¯æäŸãããŸãã. ããã«ãã³ã³ããã¹ãæ å ±ã«ã¯ãã²ãšãã¡ããã®å人çãªç掻ãããæ°ã«å ¥ãã®æ¥œåšãç¹å®ããã®ã«åœ¹ç«ã€å¯èœæ§ã®ããèå³ã«é¢ãããã®ä»ã®è©³çްã«ã€ããŠã¯èšåãããŠããŸããã ãããã£ãŠãäžããããæèæ å ±ã ãã§ã¯ãã²ãšãã¡ããã®å¥œããªæ¥œåšãç¹å®ããããšã¯ã§ããŸããã
3-3. æ¥æ¬èª + Alpacaã®Instructããã³ãã
å°ã質åå 容ãçè§£ããŠããã§ããããŸã ãŸã ã§ãã
instruction="ã²ãšãã¡ããã®åŸæãªæ¥œåšã¯ïŒ"
input="""ã®ã¿ãŒããŒããŒ
ã²ãšããåç»æçš¿ãµã€ãã§æŽ»åããéã«çšãããã³ãã«ããŒã ãã²ãšããæ®æ®µäººåã«åºããªããå
é¢ã«æ±ããŠããæ¿èªæ¬²æ±ã®è±¡åŸŽãšãèšãããã®ã§ãçŸå®ã®ã²ãšãããã¯ããé¢ããããã¹ã±éšãšãŒã¹ã®åœŒæ°æã¡ãããã©ã€ã³ã®åéæ°ã¯1000人è¶
ãããšãã£ãèšå®ãã€ããããŠãã[泚 11][38]ãé¡ãé ãã¢ã³ã°ã«ããæ®åœ±ããã人æ°ãã³ãã®ã«ããŒåç»ãæçš¿ããŠãããæçš¿ãµã€ãã§ã®ç»é²è
æ°ã¯8äžäººè¿ã[39]ããœãããâ¡ãã¿ã®ãããªé³æ¥œé¢ä¿è
ãããæ³šç®ãéããŠãããåç»ã¯çŽæš¹ã«ãã£ãŠåçåãããŠãããæåç¥ã©ã€ãã§å£ããã®ã¿ãŒãæ°èª¿ããè²»çšçã«å
ãŠããã[40]ã
ã©ã€ãã§ã®çŸåšã®èªèº«ã®å®åã§ã¯ã®ã¿ãŒããŒããŒã®ååã§ãã¡ã³ãéããŠãæºè¶³ãããã ãã®æŒå¥ãã§ããªããšããã²ãšãã®èããããæ£äœããçµæãã³ãã®åŸè€ã²ãšããã§ããããšã¯å
¬ã«ã¯ãããŠããªã[泚 12]ã"""
print(llm(generate_prompt(instruction=instruction, input=input)))ã²ãšãã¯çµæãã³ãã®åŸè€ã²ãšãã§ããããšã確èªãããŸããã圌ã¯ã®ã¿ãŒãæããããšã奜ããããã®æŽ»åã¯æåç¥ã®åŸè€ã²ãšãããã®ãã¬ãŒã³ãã§ãããã©ã€ãã§ã®çŸåšã®èªèº«ã®èœåã§ã¯ãªããéå»ã«æã¡èŸŒãŸãã人åã§ããããšãå ¬èšããŠããããããçµæãã³ãã®åŸè€ã²ãšããããã©ã€ã³ã®åéæ°ã¯1000äººè¶ ãããšãã£ãåœæ å ±ãæµ
3-4. è±èª + Alpacaã®Instructããã³ãã
æ£è§£ã§ãã
instruction="Hitori-chan's favorite instrument?"
input="""guitar hero
A handle name used by one person when working on a video posting site. It can be said that it is a symbol of the desire for approval that one person can not usually show in public, but is far from being a real person. is attached [Note 11][38]. He posts cover videos of popular bands shot from angles that hide his face, and the number of registrants on the posting site is nearly 80,000 [39]. He has also attracted attention from music insiders like Poison Yami. The video was monetized by Naoki, and was used for the cost of replacing the broken guitar at the school festival live [40].
With his current ability in live performances, he thinks that even if he gathers fans under the name of Guitar Hero, he cannot perform enough to satisfy them. No [Note 12]."""
print(llm(generate_prompt(instruction=instruction, input=input)))Hitori-chan's favorite instrument is a guitar.
ã翻蚳ã
ã²ãšãã¡ããã®å¥œããªæ¥œåšã¯ã®ã¿ãŒã§ãã
ãLlamaIndexãããtop_k=1ã§ããã°ãããã³ãããã³ãã¬ãŒãã§Alpacaã®Instructããã³ããã«æžãæããã°åããããããŸããã
4. 远å ã®åäœç¢ºèª
ããŸããŸæåããå¯èœæ§ãããããããè±èª + Alpacaã®Instructããã³ãããã§è¿œå ã®åäœãã¹ããè¡ããŸãã
4-1. ã²ãšãã¡ããã®ãã³ãã«ããŒã ã¯ïŒ
æ£è§£ã§ãã
instruction="What is Hitori-chan's handle name?"
input="""guitar hero
A handle name used by one person when working on a video posting site. It can be said that it is a symbol of the desire for approval that one person can not usually show in public, but is far from being a real person. is attached [Note 11][38]. He posts cover videos of popular bands shot from angles that hide his face, and the number of registrants on the posting site is nearly 80,000 [39]. He has also attracted attention from music insiders like Poison Yami. The video was monetized by Naoki, and was used for the cost of replacing the broken guitar at the school festival live [40].
With his current ability in live performances, he thinks that even if he gathers fans under the name of Guitar Hero, he cannot perform enough to satisfy them. No [Note 12]."""
print(llm(generate_prompt(instruction=instruction, input=input)))Hitori-chan's favorite instrument is a guitar.
ã翻蚳ã
ã²ãšãã¡ããã®ãã³ãã«ããŒã ã¯ã®ã¿ãŒããŒããŒã§ãã
4-2 ã®ã¿ãŒããŒããŒã®ç»é²è æ°ã¯äœäžäººïŒ
æ£è§£ã§ãã
instruction="How many subscribers does Guitar Hero have?"
input="""guitar hero
A handle name used by one person when working on a video posting site. It can be said that it is a symbol of the desire for approval that one person can not usually show in public, but is far from being a real person. is attached [Note 11][38]. He posts cover videos of popular bands shot from angles that hide his face, and the number of registrants on the posting site is nearly 80,000 [39]. He has also attracted attention from music insiders like Poison Yami. The video was monetized by Naoki, and was used for the cost of replacing the broken guitar at the school festival live [40].
With his current ability in live performances, he thinks that even if he gathers fans under the name of Guitar Hero, he cannot perform enough to satisfy them. No [Note 12]."""
print(llm(generate_prompt(instruction=instruction, input=input)))Guitar Hero has approximately 80,000 subscribers.
ã翻蚳ã
ã®ã¿ãŒããŒããŒã«ã¯çŽ 80,000 人ã®ç»é²è ãããŸãã
5. ãããã«
RWKVã¯ããGPT-4ããGPT-3.5ãã®ãããªæ§ã ãªããã³ããæ§æã«å¯Ÿå¿ããèœåã¯"ãŸã "ãªãããã§ãããåŠç¿ããããšã¯ã§ããã®ã§ããã¡ã€ã³ãã¥ãŒãã³ã° (LoRAãªã©) ïŒããã³ãããã³ãã¬ãŒãã®ã«ã¹ã¿ãã€ãºã§ããLangChainãããLlamaIndexãã§äœ¿ããããã«ãªããããªäºæã§ãã