
Google Colab ã§ Japanese StableLM Alpha + LlamaIndex ã® QA ã詊ã
ãGoogle Colabãã§ãJapanese StableLM Alpha + LlamaIndexãã® QA ã詊ããã®ã§ãŸãšããŸããã
ãæ³šæãGoogle Colab Pro/Pro+ ã®A100ã§åäœç¢ºèªããŠããŸãã
1. 䜿çšã¢ãã«
ä»åã¯ããjapanese-stablelm-instruct-alpha-7bã(8bitéåå)ãšåã蟌ã¿ã¢ãã«ãmultilingual-e5-largeãã䜿ããŸãã
ãJapanese StableLM Alphaãã®ã¢ãã«ã䜿çšããã«ã¯ãå©çšç³è«ãå¿ èŠã§ãã以äžã§è§£èª¬ããŠãŸãã
2. ããã¥ã¡ã³ãã®æºå
ä»åã¯ããã³ã¬ããã£ã¢ã®ããŒã£ã¡ã»ãã»ãã£ãïŒãã®ããããã®ããã¥ã¡ã³ããçšæããŸããã
ã»bocchi.txt

3. Colabã§ã®å®è¡
Colabã§ã®å®è¡æé ã¯ã次ã®ãšããã§ãã
(1) ã¡ãã¥ãŒãç·šéâããŒãããã¯ã®èšå®ãã§ããããŒããŠã§ã¢ã¢ã¯ã»ã©ã¬ãŒã¿ãã§ãGPUãã®ãA100ããéžæã
(2) ããã±ãŒãžã®ã€ã³ã¹ããŒã«ã
# ããã±ãŒãžã®ã€ã³ã¹ããŒã«
!pip install llama-index
!pip install transformers accelerate bitsandbytes
!pip install sentencepiece einops sentence_transformers(3) HuggingFaceã®ãã°ã€ã³ã
# HuggingFaceã®ãã°ã€ã³
!huggingface-cli login
_| _| _| _| _|_|_| _|_|_| _|_|_| _| _| _|_|_| _|_|_|_| _|_| _|_|_| _|_|_|_|
_| _| _| _| _| _| _| _|_| _| _| _| _| _| _| _|
_|_|_|_| _| _| _| _|_| _| _|_| _| _| _| _| _| _|_| _|_|_| _|_|_|_| _| _|_|_|
_| _| _| _| _| _| _| _| _| _| _|_| _| _| _| _| _| _| _|
_| _| _|_| _|_|_| _|_|_| _|_|_| _| _| _|_|_| _| _| _| _|_|_| _|_|_|_|
To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .
Token:
Add token as git credential? (Y/n) n
Token is valid (permission: read).
Your token has been saved to /root/.cache/huggingface/token
Login successful(4) ãã°ã¬ãã«ã®èšå®ã
import logging
import sys
# ãã°ã¬ãã«ã®èšå®
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG, force=True)(5) Colabã«dataãã©ã«ããäœæããŠããã¥ã¡ã³ããé
眮ã
巊端ã®ãã©ã«ãã¢ã€ã³ã³ã§ãã¡ã€ã«äžèЧã衚瀺ããå³ã¯ãªãã¯ãæ°ãããã©ã«ããã§dataãã©ã«ããäœæããããã¥ã¡ã³ãããã©ãã°&ããããããŸãã

(6) ããã¥ã¡ã³ãã®èªã¿èŸŒã¿ã
from llama_index import SimpleDirectoryReader
# ããã¥ã¡ã³ãã®èªã¿èŸŒã¿
documents = SimpleDirectoryReader("data").load_data()(7) LLMã®æºåã
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline
from langchain.llms import HuggingFacePipeline
import torch
# ããŒã¯ãã€ã¶ãŒãšã¢ãã«ã®æºå
tokenizer = AutoTokenizer.from_pretrained(
"novelai/nerdstash-tokenizer-v1",
use_fast=False,
)
model = AutoModelForCausalLM.from_pretrained(
"stabilityai/japanese-stablelm-instruct-alpha-7b",
load_in_8bit=True,
device_map="auto",
trust_remote_code=True,
)
# ãã€ãã©ã€ã³ã®æºå
pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
max_new_tokens=256
)
# LLMã®æºå
llm = HuggingFacePipeline(pipeline=pipe)(8) åã蟌ã¿ã¢ãã«ã®æºåã
from langchain.embeddings import HuggingFaceEmbeddings
from llama_index import LangchainEmbedding
from typing import Any, List
# queryä»ãã®HuggingFaceEmbeddings
class HuggingFaceQueryEmbeddings(HuggingFaceEmbeddings):
def __init__(self, **kwargs: Any):
super().__init__(**kwargs)
def embed_documents(self, texts: List[str]) -> List[List[float]]:
return super().embed_documents(["query: " + text for text in texts])
def embed_query(self, text: str) -> List[float]:
return super().embed_query("query: " + text)
# åã蟌ã¿ã¢ãã«ã®æºå
embed_model = LangchainEmbedding(
HuggingFaceQueryEmbeddings(model_name="intfloat/multilingual-e5-large")
)(9) ããŒãããŒãµãŒã®æºåã
ããã¹ãã¹ããªãã¿ãŒã«ãLangChainã®RecursiveCharacterTextSplitterã䜿ããŸãã
from langchain.text_splitter import RecursiveCharacterTextSplitter
from llama_index.node_parser import SimpleNodeParser
# ãã£ã³ã¯ã®åå²
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=514-7, # ãã£ã³ã¯ã®æå€§æåæ°
chunk_overlap=20, # ãªãŒããŒã©ããã®æå€§æåæ°
)
# ããŒãããŒãµãŒã®æºå
node_parser = SimpleNodeParser(text_splitter=text_splitter)(10) ãµãŒãã¹ã³ã³ããã¹ãã®æºåã
from llama_index import ServiceContext
# ãµãŒãã¹ã³ã³ããã¹ãã®æºå
service_context = ServiceContext.from_defaults(
llm=llm,
embed_model=embed_model,
node_parser=node_parser,
)(11) ã€ã³ããã¯ã¹ã®äœæã
from llama_index import VectorStoreIndex
# ã€ã³ããã¯ã¹ã®äœæ
index = VectorStoreIndex.from_documents(
documents,
service_context=service_context,
)Adding chunk: çµæãã³ã...
Adding chunk: åŸè€ã²ãšãã¯åéãäœããªãé°ãã£ã§ãã€ãäžäººã§éãããŠãããã...
Adding chunk: ããšãç¥ããçŽäœïŒããïŒæ²æã®æ«ã...
Adding chunk: æåç¥ã©ã€ã...
Adding chunk: å€äŒã¿ã«å
¥ãã...
Adding chunk: ãããããŠã²ãšããã¡ã¯ã...
Adding chunk: ãã¢å¯©æ»...
Adding chunk: æªç¢ºèªã©ã€ãªããã«åå ããããã«ã¯ã...
Adding chunk: ããçµæãã³ãã ã£ããã...
Adding chunk: åŸè€ã²ãšã(ããšãã²ãšã)...
Adding chunk: ç§è¯é«æ ¡ã«éã女åã...
Adding chunk: ãã®ã ããä»äººãšåãããã»ãã·ã§ã³ã®çµéšãçç¡ãªããã...
Adding chunk: äŒå°ç¥è¹å€(ããã¡ã«ãã)...
Adding chunk: äžåæ²¢é«æ ¡ã«éã女åã...
Adding chunk: ããŠããŠããã...
Adding chunk: å±±ç°ãªã§ãŠ(ããŸã ããã)...
Adding chunk: äžåæ²¢é«æ ¡ã«éã女åã...
Adding chunk: ãåã°ããæçžŸã¯ã€ãã«èµ€ç¹ã...
Adding chunk: åå€é代(ããããã)...
Adding chunk: ç§è¯é«æ ¡ã«éã女åã...
Adding chunk: 以äžããã...
Adding chunk: 廣äºããã(ã²ããããã)...
Adding chunk: å®å掟ãµã€ã±ããªãã¯ããã¯ãã³ããSICKHACK(ã·ãã¯ããã¯)ãã«æå±ãã女æ§ã...
Adding chunk: ã®ã·ã£ã¯ãŒã䜿ã£ãããã¿ã飯ã«ããã€ãããšããŠããããã...
Adding chunk: åŸè€ãµãã(ããšããµãã)...
Adding chunk: æž
æ°Žã€ã©ã€ã¶(ãã¿ããããã)...
Adding chunk: 岩äžå¿éº»(ããããããŸ)...
Adding chunk: é·è°·å·ããã³(ã¯ãããããã³)...
Adding chunk: äœã
æšæ¬¡å(ãããã€ãã)...
Adding chunk: 倧槻ãšãšã³(ããã€ãããã)...
Adding chunk: 人æ°ã¡ã¿ã«ãã³ããSIDEROS(ã·ããã¹)ãã®ãªãŒããŒãåããå°å¥³ã...
Adding chunk: äœè€æå(ããšãããã)...
Adding chunk: ããªãŒã©ã€ã¿ãŒãšããŠæŽ»åãã女æ§ã...
Adding chunk: ã«åå ããããã£ãããã€ãã£ãã...
Adding chunk: ã²ãšãã®ç¶(ã²ãšãã®ã¡ã¡)...
Adding chunk: 2å·(ã«ãã)...
Adding chunk: åç°é次é(ããã ããããã)...
Adding chunk: PAãã(ãŽãŒããŒãã)...
Adding chunk: äŒå°ç¥ææ(ããã¡ããã)...
Adding chunk: å®çãã³ãŽãŒä»®é¢(ãããã
ããŸãããŒããã)...
Adding chunk: ã®ã¿ãŒããŒããŒ...(12) QAãã³ãã¬ãŒãã®æºåã
LlamaIndex(ããã©ã«ã)ãLangChainã®QAãã³ãã¬ãŒãã§ã¯ããŸãåçããŠãããŸããã§ããã
from llama_index.prompts.prompts import QuestionAnswerPrompt
# QAãã³ãã¬ãŒãã®æºå
qa_template = QuestionAnswerPrompt("""以äžã¯ãã¿ã¹ã¯ã説æããæç€ºãšãæèã®ããå
¥åã®çµã¿åããã§ããèŠæ±ãé©åã«æºããå¿çãæžããªããã
### æç€º:
{query_str}
### å
¥å:
{context_str}
### å¿ç:
""")(13) ã¯ãšãªãšã³ãžã³ã®äœæã
# ã¯ãšãªãšã³ãžã³ã®äœæ
query_engine = index.as_query_engine(
similarity_top_k=3,
text_qa_template=qa_template,
)(14) 質åå¿çã
# 質åå¿ç
response = query_engine.query("åŸè€ã²ãšãã®åŸæãªæ¥œåšã¯ïŒ")åŸè€ã²ãšãã®åŸæãªæ¥œåšã¯ã®ã¿ãŒã§ãã4. 远å ã®è³ªåå¿ç
10åã®è³ªåãæããŠã¿ãŸãã
# å
¥å
inputs = [
"åŸè€ã²ãšãã®åŸæãªæ¥œåšã¯ïŒ",
"åŸè€ã²ãšãã®åйã®ååã¯ïŒ",
"åŸè€ã²ãšããå å
¥ãããã³ãåã¯ïŒ",
"ã®ã¿ãŒããŒããŒã®æ£äœã¯ïŒ",
"åå€é代ã®é«ªã®è²ã¯ïŒ",
"äŒå°ç¥è¹å€ãéãåŠæ ¡ã®ååã¯ïŒ",
"å±±ç°ãªã§ãŠã®è¶£å³ã¯ïŒ",
"廣äºããããæå±ãããã³ãåã¯ïŒ",
"ã©ã€ãããŠã¹ãSTARRYãã®åºé·ã®ååã¯ïŒ",
"ãŒã£ã¡ã¡ãããæåç¥ã§æ«é²ããæŒå¥æ³ã¯ïŒ",
]
# ãã§ãŒã³ã®å®è¡
for input in inputs:
print(query_engine.query("input))ã»åŸè€ã²ãšãã®åŸæãªæ¥œåšã¯ã®ã¿ãŒã§ãã
ã»åŸè€ã²ãšãã®åйã®ååã¯ãåŸè€ãµãã
ã»åŸè€ã²ãšããå å ¥ãããã³ãåã¯ãçµæãã³ãã
ã»åŸè€ã²ãšãã®æ£äœã¯ãã®ã¿ãŒããŒããŒã®åç»é ä¿¡è ã§ããã
ã»åå€é代(ããããã)ã®é«ªã®è²ã¯èµ€ã
ã»äžåæ²¢ã®ã©ã€ãããŠã¹ãSTARRY(ã¹ã¿ãŒãªãŒ)ããäžæ£è§£ã
ã»è¶£å³ã¯ã®ã¿ãŒã匟ãããšã§ãããäžæ£è§£ã
ã»ãSIDEROS(ã·ããã¹)ã
ã»äŒå°ç¥ææ
ã»æåç¥ã©ã€ãã§æ«é²ããæŒå¥æ³ã¯ãããã«ããã¯å¥æ³ã§ãã
æ£è§£çã¯8/10ã§ããã
ãããŸãã QAãã³ãã¬ãŒã
ã»LlamaIndexã®QAãã³ãã¬ãŒã
Context information is below.
---------------------
{context_str}
---------------------
Given the context information and not prior knowledge, answer the question: {query_str}ã»LangChainã®QAãã³ãã¬ãŒã
Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.
{context_str}
Question: {query_str}
Helpful Answer:ã»Japanese StableLM Alphaã®QAãã³ãã¬ãŒã
以äžã¯ãã¿ã¹ã¯ã説æããæç€ºãšãæèã®ããå
¥åã®çµã¿åããã§ããèŠæ±ãé©åã«æºããå¿çãæžããªããã
### æç€º:
{query_str}
### å
¥å:
{context_str}
### å¿ç: