Fine-tuning之数据质量处理

原始数据
   ↓
格式统一
   ↓
简单去重
   ↓
语义去重
   ↓
垃圾过滤
   ↓
长度控制
   ↓
语言过滤
   ↓
语义匹配检测
   ↓
困惑度过滤
   ↓
最终SFT数据

seen = set()

clean_data = []

for item in data:

    text = item["messages"][0]["content"]

    if text not in seen:
        seen.add(text)
        clean_data.append(item)

What is Python?
Explain Python.
Tell me about Python language.

计算 embedding
↓
cosine similarity
↓
>0.9 删除

bad_words = ["I don't know", "Sorry", "Maybe"]

def is_bad(text):

    for w in bad_words:
        if w in text:
            return True

    return False

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")

tokens = tokenizer.encode(text)

if len(tokens) > 4096:
    continue

用一个语言模型计算 困惑度（Perplexity）。

如果困惑度太高：

说明文本质量差。

user
assistant
user
assistant

user
user
assistant
assistant

roles = [m["role"] for m in messages]

for i in range(len(roles)-1):
    if roles[i] == roles[i+1]:
        return False

def score_data(question, answer):

    prompt = f"""
Rate the quality of this QA pair from 0-1.

Q: {question}
A: {answer}

Only output a number.
"""

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role":"user","content":prompt}]
    )

    return int(response.choices[0].message.content)

原始数据 100万
↓
清洗后 10万
↓
最终训练 5万