diff --git a/cn-Book/5.在无标记数据集上进行预训练.md b/cn-Book/5.在无标记数据集上进行预训练.md index eaa07fd..a9d5944 100644 --- a/cn-Book/5.在无标记数据集上进行预训练.md +++ b/cn-Book/5.在无标记数据集上进行预训练.md @@ -158,7 +158,7 @@ inputs = torch.tensor([[16833, 3626, 6100], # ["every effort moves", [40, 1107, 588]]) # "I really like"] # Matching these inputs, the `targets` contain the token IDs we aim for the model to produce: targets = torch.tensor([[3626, 6100, 345 ], # [" effort moves you", - [107, 588, 11311]]) # " really like chocolate"] + [1107, 588, 11311]]) # " really like chocolate"] ``` 需要注意的是,目标值中展示的是输入数据向前偏移了一个位置。我们在第 2 章实现数据加载器时已介绍过这一概念。这种偏移策略对于教会模型预测序列中的下一个 token 至关重要。