Background:A grid search, at the cost of training and testing a large number of models, is an effective way to optimize the prediction performance of deep learning models. A challenging task concerning grid search is time management. Without a good time management scheme, a grid search can easily be set off as a “mission” that will not finish in our lifetime. In this study, we introduce a heuristic three-stage mechanism for managing the running time of low-budget grid searches with deep learning, sweet-spot grid search (SSGS) and randomized grid search (RGS) strategies for improving model prediction performance, in an application of predicting the 5-year, 10-year, and 15-year risk of breast cancer metastasis.Methods:We develop deep feedforward neural network (DFNN) models and optimize the prediction performance of these models through grid searches. We conduct eight cycles of grid searches in three stages, focusing on learning a reasonable range of values for each of the adjustable hyperparameters in Stage 1, learning the sweet-spot values of the set of hyperparameters and estimating the unit grid search time in Stage 2, and conducting multiple cycles of timed grid searches to refine model prediction performance with SSGS and RGS in Stage 3. We conduct various SHAP analyses to explain the prediction, including a unique type of SHAP analyses to interpret the contributions of the DFNN-model hyperparameters.Results:The grid searches we conducted improved the risk prediction of 5-year, 10-year, and 15-year breast cancer metastasis by 18.6%, 16.3%, and 17.3%, respectively, over the average performance of all corresponding models we trained using the RGS strategy.Conclusions:Grid search can greatly improve model prediction. Our result analyses not only demonstrate best model performance but also characterize grid searches from various aspects such as their capabilities of discovering decent models and the unit grid search time. The three-stage mechanism worked effectively. It not only made our low-budget grid searches feasible and manageable but also helped improve the model prediction performance of the DFNN models. Our SHAP analyses not only identified clinical risk factors important for the prediction of future risk of breast cancer metastasis, but also DFNN-model hyperparameters important to the prediction of performance scores.
背景：网格搜索是一种优化深度学习模型预测性能的有效方法，但需要训练和测试大量模型，因此时间成本较高。网格搜索面临的一个关键挑战是时间管理。若缺乏合理的时间管理方案，网格搜索可能成为一项“无法在有限时间内完成的任务”。本研究提出一种启发式三阶段机制，用于管理低预算深度学习网格搜索的运行时间，并结合甜点网格搜索（SSGS）和随机网格搜索（RGS）策略提升模型预测性能，应用于预测乳腺癌5年、10年和15年转移风险。 方法：我们开发了深度前馈神经网络（DFNN）模型，并通过网格搜索优化其预测性能。研究采用三阶段八轮网格搜索方案：第一阶段聚焦于确定各可调超参数的合理取值范围；第二阶段探索超参数组合的甜点值并估算单位网格搜索时间；第三阶段通过多轮定时网格搜索，结合SSGS和RGS策略精细化提升模型预测性能。同时采用多种SHAP分析方法解释预测结果，包括一种独特的SHAP分析框架用于解读DFNN模型超参数对预测的贡献。 结果：相较于采用RGS策略训练的所有对应模型的平均性能，本研究实施的网格搜索将乳腺癌5年、10年和15年转移风险预测性能分别提升了18.6%、16.3%和17.3%。 结论：网格搜索能显著提升模型预测性能。我们的结果分析不仅展示了最佳模型性能，还从多个维度刻画了网格搜索特性，包括其发现优质模型的能力和单位网格搜索时间等。三阶段机制运行高效，不仅使低预算网格搜索具备可行性和可管理性，同时有效提升了DFNN模型的预测性能。SHAP分析不仅识别出对预测乳腺癌转移风险至关重要的临床风险因素，还揭示了影响模型性能评分的关键DFNN超参数。
肿瘤(癌症)患者之家