
Google Colab ã§ Vicuna-v1.5 + LlamaIndex ã® QA ã詊ã
ãGoogle Colabãã§ãVicuna-v1.5 + LlamaIndexãã® QA ã詊ããã®ã§ãŸãšããŸããã
ãæ³šæãT4ã®ãã€ã¡ã¢ãªã§åäœç¢ºèªããŠããŸãã
äž»ãªãã€ã³ãã¯ã次ã®ãšããã§ãã
ã»ã¢ãã«èªã¿èŸŒã¿æã«12GB以äžRUMæ¶è²»ãããããã€ã¡ã¢ãª
ã»Llama2ã®QAãã³ãã¬ãŒãã¯ããã©ã«ããå©çš  (Llama2ãšéã£ãŠçè§£ãã)
ã»ãæ¥æ¬èªã§åçããŠãã ããããã¯åé€ãã (Llama2ãšåæ§ã«è±èªã§è¿ã)
ã»multilingual-e5-largeã®å ¥åã«ãquery: ããä»å (粟床åäž)
1. 䜿çšã¢ãã«
ä»åã¯ããlmsys/vicuna-7b-v1.5ã(8bitéåå)ãšåã蟌ã¿ã¢ãã«ãmultilingual-e5-largeãã䜿ããŸãã
2. ããã¥ã¡ã³ãã®æºå
ä»åã¯ããã³ã¬ããã£ã¢ã®ããŒã£ã¡ã»ãã»ãã£ãïŒãã®ããããã®ããã¥ã¡ã³ããçšæããŸããã
ã»bocchi.txt

3. Colabã§ã®å®è¡
Colabã§ã®å®è¡æé ã¯ã次ã®ãšããã§ãã
(1) ã¡ãã¥ãŒãç·šéâããŒãããã¯ã®èšå®ãã§ããããŒããŠã§ã¢ã¢ã¯ã»ã©ã¬ãŒã¿ãã§ãGPUãã®ãT4ãããã€ã¡ã¢ãªããéžæã
(2) ããã±ãŒãžã®ã€ã³ã¹ããŒã«ã
# ããã±ãŒãžã®ã€ã³ã¹ããŒã«
!pip install llama-index
!pip install transformers accelerate bitsandbytes sentence_transformers sentencepiece(3) ãã°ã¬ãã«ã®èšå®ã
import logging
import sys
# ãã°ã¬ãã«ã®èšå®
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG, force=True)(4) Colabã«dataãã©ã«ããäœæããŠããã¥ã¡ã³ããé
眮ã
巊端ã®ãã©ã«ãã¢ã€ã³ã³ã§ãã¡ã€ã«äžèЧã衚瀺ããå³ã¯ãªãã¯ãæ°ãããã©ã«ããã§dataãã©ã«ããäœæããããã¥ã¡ã³ãããã©ãã°&ããããããŸãã

(6) ããã¥ã¡ã³ãã®èªã¿èŸŒã¿ã
from llama_index import SimpleDirectoryReader
# ããã¥ã¡ã³ãã®èªã¿èŸŒã¿
documents = SimpleDirectoryReader("data").load_data()(7) LLMã®æºåã
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline
from langchain.llms import HuggingFacePipeline
import torch
# ããŒã¯ãã€ã¶ãŒãšã¢ãã«ã®æºå
tokenizer = AutoTokenizer.from_pretrained(
"lmsys/vicuna-7b-v1.5",
use_fast=False
)
model = AutoModelForCausalLM.from_pretrained(
"lmsys/vicuna-7b-v1.5",
load_in_8bit=True,
torch_dtype=torch.float16,
device_map="auto",
)
# ãã€ãã©ã€ã³ã®æºå
pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
max_new_tokens=256
)
# LLMã®æºå
llm = HuggingFacePipeline(pipeline=pipe)(8) åã蟌ã¿ã¢ãã«ã®æºåã
ãintfloat/multilingual-e5-largeãã®å
¥åã®å
é ã«ãquery: ããä»å ããæ¹ã粟床ãäžãããšã®ããšãªã®ã§ä»å ããŸããã
from langchain.embeddings import HuggingFaceEmbeddings
from llama_index import LangchainEmbedding
from typing import Any, List
# queryä»ãã®HuggingFaceEmbeddings
class HuggingFaceQueryEmbeddings(HuggingFaceEmbeddings):
def __init__(self, **kwargs: Any):
super().__init__(**kwargs)
def embed_documents(self, texts: List[str]) -> List[List[float]]:
return super().embed_documents(["query: " + text for text in texts])
def embed_query(self, text: str) -> List[float]:
return super().embed_query("query: " + text)
# åã蟌ã¿ã¢ãã«ã®æºå
embed_model = LangchainEmbedding(
HuggingFaceQueryEmbeddings(model_name="intfloat/multilingual-e5-large")
)(9) ããŒãããŒãµãŒã®æºåã
ããã¹ãã¹ããªãã¿ãŒã«ãLangChainã®RecursiveCharacterTextSplitterã䜿ããŸãããã£ã³ã¯ã®æå€§æ°ããquery :ãåæžãããŠããŸãã(ããŒã¯ã³æ°ãªã®ã§7ã§ãªã3ã«ããæ¹ãè¯ããã)
from langchain.text_splitter import RecursiveCharacterTextSplitter
from llama_index.node_parser import SimpleNodeParser
# ãã£ã³ã¯ã®åå²
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=514-7, # ãã£ã³ã¯ã®æå€§æåæ°
chunk_overlap=20, # ãªãŒããŒã©ããã®æå€§æåæ°
)
# ããŒãããŒãµãŒã®æºå
node_parser = SimpleNodeParser(text_splitter=text_splitter)(10) ãµãŒãã¹ã³ã³ããã¹ãã®æºåã
from llama_index import ServiceContext
# ãµãŒãã¹ã³ã³ããã¹ãã®æºå
service_context = ServiceContext.from_defaults(
llm=llm,
embed_model=embed_model,
node_parser=node_parser,
)(11) ã€ã³ããã¯ã¹ã®äœæã
from llama_index import VectorStoreIndex
# ã€ã³ããã¯ã¹ã®äœæ
index = VectorStoreIndex.from_documents(
documents,
service_context=service_context,
)Adding chunk: çµæãã³ã...
Adding chunk: åŸè€ã²ãšãã¯åéãäœããªãé°ãã£ã§ãã€ãäžäººã§éãããŠãããã...
Adding chunk: ããšãç¥ããçŽäœïŒããïŒæ²æã®æ«ã...
Adding chunk: æåç¥ã©ã€ã...
Adding chunk: å€äŒã¿ã«å
¥ãã...
Adding chunk: ãããããŠã²ãšããã¡ã¯ã...
Adding chunk: ãã¢å¯©æ»...
Adding chunk: æªç¢ºèªã©ã€ãªããã«åå ããããã«ã¯ã...
Adding chunk: ããçµæãã³ãã ã£ããã...
Adding chunk: åŸè€ã²ãšã(ããšãã²ãšã)...
Adding chunk: ç§è¯é«æ ¡ã«éã女åã...
Adding chunk: ãã®ã ããä»äººãšåãããã»ãã·ã§ã³ã®çµéšãçç¡ãªããã...
Adding chunk: äŒå°ç¥è¹å€(ããã¡ã«ãã)...
Adding chunk: äžåæ²¢é«æ ¡ã«éã女åã...
Adding chunk: ããŠããŠããã...
Adding chunk: å±±ç°ãªã§ãŠ(ããŸã ããã)...
Adding chunk: äžåæ²¢é«æ ¡ã«éã女åã...
Adding chunk: ãåã°ããæçžŸã¯ã€ãã«èµ€ç¹ã...
Adding chunk: åå€é代(ããããã)...
Adding chunk: ç§è¯é«æ ¡ã«éã女åã...
Adding chunk: 以äžããã...
Adding chunk: 廣äºããã(ã²ããããã)...
Adding chunk: å®å掟ãµã€ã±ããªãã¯ããã¯ãã³ããSICKHACK(ã·ãã¯ããã¯)ãã«æå±ãã女æ§ã...
Adding chunk: ã®ã·ã£ã¯ãŒã䜿ã£ãããã¿ã飯ã«ããã€ãããšããŠããããã...
Adding chunk: åŸè€ãµãã(ããšããµãã)...
Adding chunk: æž
æ°Žã€ã©ã€ã¶(ãã¿ããããã)...
Adding chunk: 岩äžå¿éº»(ããããããŸ)...
Adding chunk: é·è°·å·ããã³(ã¯ãããããã³)...
Adding chunk: äœã
æšæ¬¡å(ãããã€ãã)...
Adding chunk: 倧槻ãšãšã³(ããã€ãããã)...
Adding chunk: 人æ°ã¡ã¿ã«ãã³ããSIDEROS(ã·ããã¹)ãã®ãªãŒããŒãåããå°å¥³ã...
Adding chunk: äœè€æå(ããšãããã)...
Adding chunk: ããªãŒã©ã€ã¿ãŒãšããŠæŽ»åãã女æ§ã...
Adding chunk: ã«åå ããããã£ãããã€ãã£ãã...
Adding chunk: ã²ãšãã®ç¶(ã²ãšãã®ã¡ã¡)...
Adding chunk: 2å·(ã«ãã)...
Adding chunk: åç°é次é(ããã ããããã)...
Adding chunk: PAãã(ãŽãŒããŒãã)...
Adding chunk: äŒå°ç¥ææ(ããã¡ããã)...
Adding chunk: å®çãã³ãŽãŒä»®é¢(ãããã
ããŸãããŒããã)...
Adding chunk: ã®ã¿ãŒããŒããŒ...(12) ã¯ãšãªãšã³ãžã³ã®äœæã
# ã¯ãšãªãšã³ãžã³ã®äœæ
query_engine = index.as_query_engine(
similarity_top_k=3,
)(13) 質åå¿çã
qa_chain.run("æ¥æ¬èªã§åçããŠãã ãããåŸè€ã²ãšãã®åŸæãªæ¥œåšã¯ïŒ")åŸè€ã²ãšãã®åŸæãªæ¥œåšã¯ã®ã¿ãŒã§ãã4. 远å ã®è³ªåå¿ç
ã»10åã®è³ªå
# å
¥å
inputs = [
"åŸè€ã²ãšãã®åŸæãªæ¥œåšã¯ïŒ",
"åŸè€ã²ãšãã®åйã®ååã¯ïŒ",
"åŸè€ã²ãšããå å
¥ãããã³ãåã¯ïŒ",
"ã®ã¿ãŒããŒããŒã®æ£äœã¯ïŒ",
"åå€é代ã®é«ªã®è²ã¯ïŒ",
"äŒå°ç¥è¹å€ãéãåŠæ ¡ã®ååã¯ïŒ",
"å±±ç°ãªã§ãŠã®è¶£å³ã¯ïŒ",
"廣äºããããæå±ãããã³ãåã¯ïŒ",
"ã©ã€ãããŠã¹ãSTARRYãã®åºé·ã®ååã¯ïŒ",
"ãŒã£ã¡ã¡ãããæåç¥ã§æ«é²ããæŒå¥æ³ã¯ïŒ",
]
# ãã§ãŒã³ã®å®è¡
for input in inputs:
print(query_engine.query("æ¥æ¬èªã§åçããŠãã ããã" + input))ã»åŸè€ã²ãšãã®åŸæãªæ¥œåšã¯ã®ã¿ãŒã§ãã
ã»åŸè€ã²ãšãã®åйã®ååã¯åŸè€ãµããã§ãã
ã»åŸè€ã²ãšããå å ¥ãããã³ãåã¯ãçµæãã³ããã§ãã
ã»ã®ã¿ãŒããŒããŒã®æ£äœã¯åŸè€ã²ãšãã§ãã
ã»åå€é代ã®é«ªã®è²ã¯ããŠã§ãŒãã®å ¥ã£ãèµ€ã髪ãã»ããã³ã°ã«äŒžã°ããŠããŸãã
ã»äŒå°ç¥è¹å€ãéãåŠæ ¡ã®ååã¯ãSTARRY(ã¹ã¿ãŒãªãŒ)ãã§ãããäžæ£è§£ã
ã»å±±ç°ãªã§ãŠã®è¶£å³ã¯ãã®ã¿ãŒã匟ãããšã§ãããäžæ£è§£ã
ã»å»£äºããããæå±ãããã³ãåã¯ãSIDEROS(ã·ããã¹)ãã§ãã
ã»äŒå°ç¥ææ(ããã¡ããã)ã§ãã
ã»ãããã£ã¡ãããæåç¥ã§æ«é²ããæŒå¥æ³ã¯ãããã«ããã¯å¥æ³ã§ãã
æ£è§£çã¯8/10ã§ããã
ã»ãæ¥æ¬èªã§åçããŠãã ããããªãã®è³ªå
ã»åŸè€ã²ãšãã®åŸæãªæ¥œåšã¯ã®ã¿ãŒã§ãã
ã»åŸè€ã²ãšãã®åйã®ååã¯åŸè€ãµããã§ãã
ã»åŸè€ã²ãšããå å ¥ãããã³ãåã¯ãçµæãã³ããã§ãã
ã»ã®ã¿ãŒããŒããŒã®æ£äœã¯åŸè€ã²ãšãã§ãã
ã»åå€é代ã®é«ªã®è²ã¯ïŒãäžæ£è§£ã
ã»äŒå°ç¥è¹å€ãéãåŠæ ¡ã®ååã¯ãSTARRY(ã¹ã¿ãŒãªãŒ)ãã§ãããäžæ£è§£ã
ã»å±±ç°ãªã§ãŠã®è¶£å³ã¯ïŒ
å±±ç°ãªã§ãŠã®è¶£å³ã¯ã®ã¿ãŒã匟ãããšã§ãããäžæ£è§£ã
ã»å»£äºããããæå±ãããã³ãåã¯ãSIDEROS(ã·ããã¹)ãã§ãã
ã»ã©ã€ãããŠã¹ãSTARRYãã®åºé·ã®ååã¯äŒå°ç¥ææã§ãã
ã»"ãŒã£ã¡ã¡ãã" who is a high school student and a guitarist. They are part of a band and are preparing for a live performance at a cultural festival. However, they have no experience playing with others and are struggling to play as a band. Despite this, they are determined to succeed and practice hard. During the cultural festival, they encounter various problems, including equipment failure, but they manage to perform well and receive applause from the audience. However, their strange behavior, such as diving into the audience, attracts attention and becomes a topic of discussion among the media.ãäžæ£è§£ã
æ£è§£çã¯6/10ã§ããã