Furthermore, when working with financial data, we can bring practitioners’ knowledge of markets and financial data to bear on our validation procedures. We know that since markets are competitive, factors decay over time; signals that may have worked well in the past may no longer work well by the current time. For this reason, we should generally test and validate on the most recent data possible, as testing on the recent past could be considered the most demanding test.
此外，在处理财务数据时，我们可以使从业人员对市场和财务数据的了解可用于我们的验证程序。我们知道，由于市场竞争激烈，因此因素会随着时间而衰减。过去可能效果良好的信号可能在当前时间不再有效。因此，我们通常应该对最新数据进行测试和验证，因为对最近历史的测试可能被认为是最苛刻的测试。
It’s possible that the design of the model may cause it to perform better or worse in different market regimes; so the most recent time period may not be in a market regime in which the model would perform well. But generally, we still prefer to use most recent data to test if the model would work in the time most similar to the present. In practice, of course, before investing a lot of money in a strategy, we would allow time to elapse without changing the model, and test its performance with this true out-of-sample data: what’s known as “paper trading”.
模型的设计可能会导致它在不同的市场体制下表现更好或更差。因此，最近的时间段可能不在该模型可以正常运行的市场体制中。但总的来说，我们仍然倾向于使用最新数据来测试该模型在与当前时间最相似的时间内是否可以正常工作。当然，实际上，在实践中，在为策略投入大量资金之前，我们会花些时间而不更改模型，并使用此真实的样本外数据（即所谓的“纸面交易”）测试其性能。
In summary, most common practice is to keep a block of data from the most recent time period as your test set.
总之，最常见的做法是将最近一段时间内的数据作为测试集。
Then, the data are split into train, valid and test sets according to the following schematic:
然后，根据下图将数据分为训练集，验证集和测试集：
When working with data that are indexed by asset and day, it’s important not to split data for the same day, but for different assets, among sets. This would manifest as a subtle form of lookahead bias. For example, say data from Coca-Cola and Pepsi for the same day ended up in different sets. Since they are very similar companies, one might expect their share price trends to be correlated. If the model were trained on data from one company, and then validated on data from the other company, it might “learn” about a price movement that affects both companies, and therefore have artificially inflated performance on the validation set.
当使用按资产和日期索引的数据时，重要的是不要在同一天中将同一种资产的数据分到一组，而是将不同资产的数据分到一组内。 这将表现为超前偏差的微妙形式(译注:某种程度上像是利用了未来数据)。 例如，说来自可口可乐和百事可乐的同一天的数据以不同的集合结束。 由于它们是非常相似的公司，因此人们可能希望它们的股价趋势相互关联。 如果模型是根据一个公司的数据进行训练的，然后根据另一公司的数据进行验证的，则它可能会“了解”会影响两家公司的价格变动，因此会人为地夸大验证集上的绩效。
Source/来源: AI for Trading, Udacity
发表回复