1.1.

ML Employment in IC Layout Applications

Wide varieties of ML applications have been incorporated throughout modern chip design and fabrication flows, in both the IC design and manufacturing phases. Our review in this study focuses on ML applications that use features derived from IC layout pattern representations.

1.2.

ML Modeling Approaches for IC Layouts

A general perspective on ML learning approaches was highlighted by Lin et al.²⁶ Considering the reviewed ML applications that use IC layouts in the previous Sec. 1.1, we reintroduce this perspective with slight modification. ML modeling can follow one of two schemes: (a) traditional approach or (b) deep-learning approach, as shown in Fig. 1. In the traditional approach, the input data represent extracted features from the IC layout, and these features are passed to a predefined architected off-the-shelf ML model from libraries, such as SciKit learn.²⁷ The main challenges with this approach are (1) choosing an adequate IC layout representation, (2) extracting descriptive features, (3) selecting a suitable off-the-shelf ML model that fits properly with the learning task, and eventually (4) tuning the model’s parameters to achieve the required accuracy in training and testing phases and avoid model overfitting. This approach provides a relatively shorter model development turnaround-time (TAT); however, it needs cautious, careful selections throughout the flow. Applications such as Refs. 2 and 8 in IC design phase, and Refs. 3, 14, 16, 23, and 26 in the IC manufacturing phase follow this approach.

Fig. 1

ML modeling approaches: (a) traditional ML approach and (b) deep-learning approach.

In the deep-learning approach, the input IC layout patterns representations are passed to a deep-learning ML model from libraries, such as TensorFlow.²⁸ In this case, the model contains an automated self-features extraction process in addition to the learning process. The self-features extraction is carefully designed and architected in compromise with the learning process’s objectives to extract meaningful features contributing to the model accuracy and avoid overfitting in regression-based learning and false-positives in classification-based learning. For example, in litho HS detection applications, Shin et al.¹¹ presented self-feature extraction through CNN deep-learning. The input layout is parsed into pixelized images through a sliding scanning window. Scanned images are passed to CNN with four convolutional pools for features extraction. The numbers of connections per image in the four convolutional pools are $1.6\times {10}^8$, $1.5\times {10}^9$, $8.6\times {10}^8$, and $1.5\times {10}^9$, respectively, and must be run with GPU assistance to accelerate the convolutional operations. Yang et al. in Refs. 10 and 19 presented deeper CNNs architectures for feature extraction and also ran with GPU assistance. Selvam et al.⁷ used density-grid pixels vectors with deep-learning CNN run with GPU assistance as well. Matsunawa et al.¹⁵ approached the feature extraction using density-pixels and DNN, where the input layout is parsed into clips, which are passed to DNN with an input layer (3600: $60\times 60$ nodes) and four hidden layers: (196: $14\times 14$ nodes), (196: $14\times 14$ nodes), (144: $12\times 12$ nodes), and (144: $12\times 12\mathrm{nodes}$), respectively. Specifying the appropriate DNN architecture is an iterative trials process¹⁵ objective by minimizing the loss function based on the number of used hidden layers, hyperparameters, transfer functions, and network connections. The main challenges with the deep-learning approach are (1) selecting the proper architecture for the deep-learning model, as there is no global solution that can fit all applications, (2) selecting the proper convolutional kernel sizes for CNN models and the number of hidden layers and the number of nodes per layer for DNN models, (3) selecting the model’s activation functions and tuning the hyperparameters to achieve the optimum accuracy, and (4) selecting proper layers connections and dropout ratios between layers to avoid model overfitting. All these steps are experimental and impact the model development TAT. In general, the optimized input clip size and pattern representation play a significant role in the accuracy of the deep-learning models. Moreover, finding the appropriate compromised architecture with proper hyperparameters of the self-features extraction process and the learning process is a significant material of research that might also result in longer model development TAT. The deep-learning approach can be traced in applications, such as Refs. 4 and 7 in the IC design phase, and Refs. 10 to 13, 15, 18, and 19 in the IC manufacturing phase.

In this study, we focus on efficient IC pattern representation with predefined architected off-the-shelf ML models to avoid deep-learning’s self-features extraction that requires high computational resources, such as GPU assistance and longer model development TAT.

1.3.

IC Layouts Feature Representation for ML Applications

Many pattern representations have been introduced to extract ML features for IC layout patterns. Although fragment-based geometrical features^2,3 provide geometrical information about the context around a fragment of interest, this approach generates a massive amount of data per fragment for a full chip layout that requires equally massive computational and storage resources. Density-based features^7,8,14,15 are one-to-many geometrical encoding representations, as one density value can be mapped to different geometrical shapes, which drops some of the spatial data about the context around a point of interest (POI) based on the selected resolution. To capture dense geometrical information, especially with tight layout dimensions, high resolution with a small grid size should be used, which may require a deep-learning approach with self-features extraction capabilities.^7,15 Although Matsunawa et al.¹⁴ presented an optimized approach in selecting adequate density grid pixels representation (grid area $1.2\mu \mathrm{m}$ and grid size $10\times 10\text{pixels}$) for litho HS detection without deep-learning by measuring the distance between HS and NHS patterns in a lower dimensional space using PCA components, this representation is bounded by a classification problem. For a regression-based problem, we may need higher resolution and accordingly, a larger number of features that may require deep-learning. CCAS was first introduced by Matsunawa et al.²¹ and applied on a fragmentation basis with a hierarchical Bayes model (HBM) to reduce the number of iterations in model-based OPC. It considers the fact that diffracted light from the mask patterns is propagating concentrically. CCAS was used in Refs. 23 and 24 for SRAF placement and in Ref. 16 for litho HS detection. Although CCAS provides a compact IC layout patterns representation, such as density-based features, it still drops some of the spatial data about the context around the POI, especially with high resolution representation. Increasing the number of sampling points creates adjacent circles with redundant information. This problem was highlighted by Geng et al.,²⁴ who tried to eliminate this redundancy through self-adaptive dictionary learning in a supervised feature encoding step, where CCAS features are mapped into a different lower dimensional space as sparse encoded features to eliminate the redundancy. The main problem with this technique is data leakage,²⁹ where the feature encoding’s objective function jointly uses the supervised predicted labeled data to encode features and then passes a piece of prediction data to the ML model’s input encoded features. In Ref. 16, Zhang et al. tried to optimize the selection of circles index using mutual information to compute the dependency between circle indexes and classification prediction variable. The training and testing data are extracted using this optimized circle indexes, which can also be considered as a source of data leakage, as the testing data set has prior knowledge about the optimized circle index. Image-pixel-based features^{4,5,10,11,20,25} do not provide a direct mapping to features, but need deep-learning models with self-features extraction. This approach is usually employed with CNN models and potentially needs GPU assistance. It should be highlighted that both density-based-pixels and image-based-pixels feature representation quality directly depend on selecting adequate resolutions for the layout critical dimensions (CD) to maintain descriptive details. Moreover, the information gain in every pixel is very high and that challenges features reduction and features merging or synthesis without deep-learning self-feature extraction. Frequency domain/DCT features¹³ are not computationally friendly—layout patterns are clipped and subjected to density-based scanning, then transformed into the frequency domain using DCT. Similarly, aerial image features¹⁷ and PFT features²² need optical simulations to be calculated, which may be suitable for ML-OPC and litho HS detection applications where features calculation is bounded by the downstream application and embedded within the overall flow. Adaptive-squish-pattern^18,19 is an adaptive-grid bitmap representation for a layout pattern that provides a compact lossless representation of layout patterns. However, it does not provide direct one-to-one mapping with geometrical features, but requires a deep-learning feature extraction through CNN models as presented in Refs. 18 and 19. Although Ref. 9 presented a mixture of density features and localized extracted geometrical features to represent layout patterns, the list of extracted features still ignores several descriptive details about the contextual patterns, such as corners and internal and external spacings. To address this problem, we present a novel edge-based approach named geometrical positioning surveying (GPS)³⁰ to represent the layout POIs and their surrounding context of patterns. Further details are discussed in Sec. 2.

1.4.

Line-End-Pull-Back Modeling Aspects

The LEPB problem can be characterized by two main aspects: (1) LE corner rounding and (2) LE shortening. Based on Fraunhofer diffraction patterns and Fourier optics,³¹ the band limitations of the optics system filter out high spatial frequencies and impose rounding at the corners³²; in LEs, this rounding is severe. LE shortening depends on the line width and surrounding context— isolated or semi-isolated LEs whose width is near the resolution limit of the litho process typically suffer from noticeable shortening on the wafer,³¹ whereas very dense LEs tend to merge with neighbors. Furthermore, the etching process exacerbates the impact of corner rounding and LE shortening. Consequently, fabrication process variation sensitivity affects both LE corner rounding and LE shortening. Figure 2 shows the difference in corners and LEs between (a) layout designed patterns and (b) simulated patterns through a litho (optical and etching) manufacturing system.

Fig. 2

Differences in corners and LEs between: (a) layout designed patterns and (b) layout lithography simulated manufactured patterns.

In the conventional approach, LEPB modeling is conducted as a part of OPC modeling³² and is objective by an edge-placement-error convergence at LEs, which mandates careful tuning for LE fragments in the OPC recipes. ML-OPC^20–22,33 and GPU-based OPC³⁴ have been proposed to accelerate OPC modeling. LEPB amounts can be precisely characterized through calibrated model-based litho (optical and etching) simulations, which require high computational resources consumption. Shin et al.¹¹ estimated that model-based litho simulation usually takes more than $100\mathrm{CPU}\mathrm{h}/{\mathrm{mm}}^2$, and it could take several days to prepare accurate calibrated models. We explain the conventional litho simulation flow in detail in Sec. 3.2.

In this paper, we present fast and accurate LEPB modeling through a supervised regression-based ML approach. Input features were approached using density pixels, density CCAS, and GPS, which is a novel edge-based approach that describes an LE as a POI and its surrounding context of patterns in the IC layout. GPS provides a direct one-to-one mapping between input features passed to the ML model and physical measurements between the POI and its surrounding patterns. The remaining sections are organized as follows: Sec. (2) explains GPS, Sec. (3) explains the conventional litho simulation flow used to model LEPB and data collection processes, Sec. (4) demonstrates the ML modeling flow, Sec. (5) demonstrates the experiments conducted on an industrial test case, and Sec. (6) presents our conclusions.

2.1.

Constrained Generality Concept in IC Layouts

The functionality of GPS relies on the constraint generality concept (CGC), which is inherent to IC layout designs. In essence, the CGC ensures that patterns in a layout design are not unconstrained, such as in a freeform drawing that may take any shape. Layout designers are restricted to certain types of geometries, such as polygons, as well as being subject to other restrictions derived by place and route tools, DRC and DFM rules, etc., all of which imply constraints in polygon width, spacing, direction, etc. These constraints not only result in layout design patterns that are a much smaller subset of the freeform population but also support the ability of the edge-based engine to describe an edge’s contextual relations and measurements to neighboring polygons.

2.2.

GPS Measurement Constructors

As shown in Fig. 3, a central POI is placed at the center of the LE tip of a residential polygon surrounded by contextual polygons in all directions and with different structures and dimensions.

Fig. 3

Layout POI and surrounding context of patterns.

A respective edge is defined based on two main features: topological features and dimensional features.³⁰ Topological features comprise quantized values restricted to a discrete set that describes a certain geometrical state about the edge, such as orientation (horizontal, or vertical) and corner type (convex, concave, or no corner). Dimensional features represent continuous numerical measurements, such as spaces, widths, enclosures, and angles. Thus, topological features are based on the conditional constraints of the layout design process, and dimensional features expound on the numerically constrained particulars of a given layout design. Consequently, decomposing the quantifiers of edges (which are the basic constructor of patterns) into topological and dimensional features requires GPS to extract direct one-to-one directional geometrical features about a POI and its surrounding context of polygons in the layout.

This description is decomposed into the following constructors:

As shown in Fig. 4, the combinational embodiments structured by these constructors form three categories of descriptive geometrical measurements:

Fig. 4

GPS geometrical measurements constructors.

2.3.

DGKs and GPS Features Vector

DGKs are the core geometrical measuring functions applied per direction in each zone. These measuring functions use the constructors shown in Fig. 4 to calculate POI, cardinal, and ordinal measurements, as explained in the previous section. These core measuring functions are implemented using Calibre SVRF commands that physically extract geometrical measured features. For example, the EXTERNAL command is used to measure distance-related features, and the INTERNAL command is used to measure width-related features. Directional, orientation, corners, and zone relations are implemented using DFM PROPERTY and DFM Functions operations. Then, the measured geometrical features by DGKs are concatenated using DFM PROPERTY commands in a single feature vector entry per each POI. Accordingly, the POI feature vector can be represented by the following equation to reflect the three categories of measurements shown in Fig. 4:

Features vector of the POI and its surrounding context of patterns are given as

Both $⟨f_{\mathrm{CC}}⟩$ and $⟨f_{\mathrm{CO}}⟩$ can be decomposed into $\{D_{\mathrm{CC}}\}$, $\{T_{\mathrm{CC}}\}$ and $\{D_{\mathrm{CO}}\}$, $\{T_{\mathrm{CO}}\}$, respectively. Where $\{D_{\mathrm{CC}}\}$, $\{D_{\mathrm{CO}}\}$ represent matrices of dimensional features for the contextual patterns in the cardinal and ordinal directions with the size of (7 features × 4 directions × 3 zones) and (10 features × 4 directions × 3 zones), respectively, $\{T_{\mathrm{CC}}\}$ and $\{T_{\mathrm{CO}}\}$ represent matrices of topological features for the contextual patterns in the cardinal and ordinal directions with the size of (3 features × 4 directions × 3 zones) and (5 features × 4 directions × 3 zones), respectively.

This topological and dimensional features decomposition expands Eq. (1) into the following equation:

Expanded features vector with dimensional and topological features are given as

2.4.

GPS Topological and Dimensional Features

Based on Eq. (2) for $V(F_{\mathrm{POI}})$, the collected geometrical features of the POI and its surrounding context of patterns can be categorized into

The GPS topological and dimensional signatures are unlike adaptive squish pattern signatures,^18,19 which are low-level bitmap descriptions of patterns where each bit represents a unique value by its own and requires further feature extraction. GPS signatures directly provide descriptive features that reflect actual geometrical (topological and dimensional) properties. Table 3 shows examples of GPS measured features with features description and feature types. Illustrative examples for the measured features are shown in Fig. 5 for the same pattern categorized as (a) POI measurements, (b) cardinal measurements, and (c) ordinal measurements. Further details about GPS features and DGKs implementation can be found in Ref. 30.

Table 3

Example of GPS measured features.

Fig. 5

Illustrative examples of GPS measured features. (a) POI measurements, (b) cardinal measurements, and (c) ordinal measurements.

Metaphorically, GPS is an edge-based camera that snaps geometrical edge-based images for the POI and its surrounding context of patterns, rather than the conventional pixel-based or density-based images.

3.1.

POI Definition and Placement

The primary objective of this study is LEPB prediction for LEs that have a nearby via, as these are the most critical LEs in any design, and the most likely locations where a systematic defect can occur that results in partial or total metal-via disconnection. Accordingly, we set the following criteria to select POIs as LE tip with length $\le 70\mathrm{nm}$ and has a via (either V1, V2, or both) placed within a distance $\le 100\mathrm{nm}$ away from the LE tip.

3.2.

Lithographic Manufacturing Simulation Flow

The conventional litho manufacturing simulation workflow is shown in Fig. 6 and the associated processed layers are shown in Fig. 7. The flow is used to precisely calculate the amounts of pullbacks and extensions at each LE, which will be used to train and test the ML models in the following sections.

Fig. 6

Model-based lithographic manufacturing workflow to measure target LEPB.

Fig. 7

Lithographic manufacturing workflow associated layers. (a) Origin drawn target V1-M2-V2, (b) placed POIs on selected line-ends, (c) etching biased retarget M2 layer, and (d) model-based OPC and SRAFs placement.

The input design layout passed the physical verification sign-off process, and input design layers (V1-M2-V2) are shown in Fig. 7(a). The POI placement criteria stated in Sec. 3.1 is applied on a testcase of size 1 mm-by-1 mm, resulting in 248,798 POIs. The placed POIs are shown in Fig. 7(b). The layout M2 layer is then subjected to etch-biasing compensation to generate the retarget layer shown in Fig. 7(c). Hybrid SRAF insertion is used to place assist features using both (1) a model-based approach using calibrated litho model (optical and etching models) and (2) rule-based approach based on design dimensions. The OPC process is performed based on a litho model basis and applied on the placed SRAFs seeds and M2 retarget layers, including (1) process-window OPC, (2) SRAF print avoidance, and (3) mask rules constraints to maintain a practical industrial approach. The placed SRAFs and OPC final masks are shown in Fig. 7(d). Generated masks (OPC and SRAFs) are used in model-based litho (optical and etching) simulation and verification to generate the final contours after etching the printed contours at nominal process window conditions. Target LEPB is measured between the final etched contours and M2 input target layer (original drawn) at POIs.

Figure 8(a) shows a histogram of target LEPB amounts (nm), where negative amounts refer to LE pullback and positive amounts refer to LE extensions. Although the maximum LEPB amount is 21.9 nm, the LEPB amount should not be viewed as a standalone value, but as a percentage of the TIP-to-VIA distance to reflect the severity of the pullback, as shown in Fig. 8(b). The maximum target LEPB severity is 42%, with a target LEPB amount of 21 nm and TIP-to-VIA distance of 50 nm.

Fig. 8

Histograms of (a) target LEPB amount (nm) and (b) target LEPB severity amount (%).

3.3.

Layout ML Features Harvesting

As explained in Sec. 3.2, the LEPB amount at each POI is measured through a conventional litho simulation flow, as shown in Fig. 6. To quantify the geometrical features provided by GPS, we integrated density pixels and density CCAS features as alternative comparative approaches. For consistency comparison between the three implemented approaches, we impose a POI-based analysis and center the SD of each approach around the POI and select adequate resolution within a comparable number of features. The optical system used in this study is a deep-ultraviolet system with an OD of $1.28\mu \mathrm{m}$; accordingly, the SD centered by the POI should be $0.640\mu \mathrm{m}$ for all approaches. It should be highlighted that in typical OPC modeling, especially etching process considerations, the SD can be beyond the OD up to $25\mu \mathrm{m}$ or even $50\mu \mathrm{m}$ to account for long-range effects. However, for consistent comparisons, we set the SD approximately equal to the system’s OD for all approaches.

3.3.1.

GPS features vector

As shown in Eq. (2), the number of measurement zones has a direct impact on the length of the features vector produced by GPS. Increasing the number of measurement zones will accordingly increase the length of the features vector. In this study, we empirically set three measurement zones with equal steps covering the SD centered by the POI up to the OD. This yields 313 features representing the POI and its surrounding context of patterns. Figure 9 shows the distribution between topological and dimensional features, and their counts for: (1) POI and resident polygon features, (2) zone 1 features, (3) zone 2 features, and (4) zone 3 features.

Fig. 9

GPS topological and dimensional features vectors and their counts.

Although the number of edges and corners of the context around each POI is different, the extracted GPS features vector length is fixed for all POIs. Figure 10(a) shows the GPS representation of the POI and its surrounding context of patterns.

3.3.2.

Density CCAS features vector

We implanted the density CCAS representation with 16 sampling circles equally distributed with a radius step of 40 nm (equal to target layer line/space width) covering the search area and 16 sampling axes with a radial angular step of ($\pi /8$), as shown in Fig. 10(b). We approached sampling points based on 1 nm square pixel in a grid of $1280\times 1280\text{pixels}$ covering the SD centered by the POI. This approach results in 16 sampling points for each sampling circle at the intersection points between circles and axes. Eventually, this sampling yields 257 density CCAS features.

Fig. 10

Incorporated layout ML features: (a) GPS, (b) density CCAS, and (c) density pixels.

4.1.

Decision-Trees and IC Layouts

The decision tree (DT) regressor is a supervised ML algorithm that aims to model a continuous prediction target by building a tree structure consisting of decision nodes and branches. A simple explanation of DT is shown in Fig. 11. The decision node contains a subset of the input features, and the decision branches are rules applied to the node’s features. Based on the number of features in a parent node and applied decision rules, subsequent child nodes are created through splits. Each of the child nodes inherits a subset of the parent node features with more pivotal branches. In the DT classifier, splits are based on information gain and entropy, whereas in the DT regressor, the split decision is based on information gain and error measurement criteria to evaluate the deviation from the prediction target.

Fig. 11

DT growing with level and leaf.

As we explained in Sec. 2.1, the IC creation process is constrained by many rules derived by place and route tools, DRC and DFM rules, etc., all of which imply constraints in polygon width, spacing, direction. Each of these constraints has boundaries and can be constructed in the form of a DT model. Dawar et al.¹⁷ charted an example of a DT model for a simple constrained rule of tip-to-tip. We reintroduce this example in Fig. 12, with slight modifications to explain how a simple IC constrained rule construction can be approached through DT models.

Fig. 12

Example of simple DT with simple constrained rule.

This analogy can be extended to GPS features as well, as the features directly represent directional topological and dimensional measurements for a POI and its surrounding context of patterns and can be constructed in the form of a DT model. In addition, DT models have been incorporated with density pixels features,¹⁴ aerial image, and geometrical features¹⁷ for litho HS detection applications, and with CCAS features for SRAF placement.²³

Initially, we selected random forest (RF)³⁵ and adaptive-boosting RF (AdaBoostRF)³⁶ ML models from Scikit-learn ensemble regressors.³⁷ However, we faced a limitation with the RF model, as it does not natively support categorical features, and GPS topological features are categorical features. We tried to use one-hot-encoding (OHE)³⁸ to encode GPS topological features, but this approach significantly increased the number of features, because based on the number of values that each categorical feature can take, OHE extends each categorical feature into several extended encoding features with low cardinality, which tends to create unbalanced biased trees.³⁹ Increasing the number of features mandated the usage of a large number of estimators that roughly fit with input data and make the model more vulnerable to overfitting, especially with AdaBoostRF. Moreover, we discovered that the GPS topological features were located at the bottom of the feature-importance list produced by the RF. This low priority meant the model did not make significant use of the benefits of GPS topological features, due to the lack of native support of categorical features by the RF regressor combined with the low cardinality of GPS topological features after OHE. Other encoding techniques such as target encoding⁴⁰ could be used, but they will make the ML modeling process subject to data leakage,²⁹ whereas the lack of native categorical features support in the RF model remains a challenge. These limitations motivated us to employ a LightGBM⁴¹ model.

4.2.

LightGBM Regressor

The LightGBM⁴¹ regressor is a gradient-boosting DT-supervised ML model that natively supports integer-encoded categorical features by finding the optimal splits over categorical features. A sorted histogram for each categorical feature’s values is used to partition the categories into subsets,⁴¹ and a reduced complexity algorithm⁴² is used to find the optimum partitions. Then, the optimal split points are located on the sorted histogram based on the training objectives at each split.³⁹ LightGBM provides two main advantages: (1) gradient-based one-side sampling, which samples the training data on a gradients-basis, and (2) exclusive feature bundling through a greedy algorithm, which provides better performance than OHE.⁴¹ One of the main advantages of LightGBM is that it is fast⁴¹ compared to other DT models, such as XGBoost.⁴³ This speed is due primarily to growing fewer trees through optimized leaf-wise splits, rather than level-wise growth, as shown in Fig. 13,⁴⁴ which significantly speeds up the training process. LightGBM also supports ensemble learning, where a series of base DT models are trained in sequence on versions of the training data, with each model achieving improved learning based on the prediction error (PE) in the previous model. The final prediction is produced through voting across all base models.

Fig. 13

DT growing through splits: (a) tree growth through leaf-wise splits (LightBGM) and (b) tree growth through level-wise splits.

A list of LightGBM model parameters that we encountered by tuning throughout our evaluation experiments is found in Table 8.

Table 8

LightGBM model parameters.

4.3.

GPS Topological Signatures Features

As explained in Sec. 2.2, GPS topological features are categorical features that take finite-state discrete data values to represent the topological aspects of the pattern. The combination of all topological features within a measurement zone is the zone’s topological signature, which is a dimensionless integer-encoded number that refers to a specific constellation of topological combinations within the zone. Patterns that have the same topological signature of a given zone have the same topological aspects in this zone. Consequently, the patterns that have the same topological signatures in all zones have the same topological feature values, and accordingly the same topological aspects across all measurement zones, although they can have the same or different dimensional features. Instead of passing all 104 GPS topological features to the LightGBM model, we use only four topological signatures features for (1) POI and resident polygon, (2) zone 1, (3) zone 2, and (4) zone 3, as shown in Fig. 14. This reduces the GPS features vector length to 213 features instead of 313 features, and, as highlighted in Sec. 4.1, the LightGBM model handles categorical features with integer-encoding values without numerical biasing.

Fig. 14

GPS features with topological signature features.

4.4.

Experimental Evaluations

Besides collecting measured target LEPB at each POI, the following features are collected: (1) localized features, (2) GPS features, (3) density-pixels features, and (4) density-CCAS features. Three experimental sets of features are formulated, as shown in Table 9. Each of the experimental feature sets is trained with LightGBM to predict target LEPB.

Table 9

ML experimental evaluations features sets.

The ML modeling flow shown in Fig. 15 starts by dividing the input data randomly into training and testing data sets, with allocations of 80% and 20%, respectively, to maintain healthy testing on unseen data by the ML models during the training phase to avoid data leakage.²⁹ The data preprocessing step normalizes input features and target prediction before starting the training phase, and the same preprocessing is applied to the testing data set. The training phase is then triggered, producing training predictions, and the calibrated ML model is passed to the testing block, which produces testing predictions. The error calculation and error analysis steps (which are explained in detail in the next section) are applied to the training and testing predictions and the results compared against the actual measured target LEPBs to determine the quality of the training and testing.

Fig. 15

ML data modeling flow.

5.1.

ML Model Error Quantifiers

The PE at each data point is defined as ${\mathrm{PE}}_i=(y_i-f(x_i))$, where $i$ represents the data point index, $y_i$ represents the actual measured target value at the data point, and $f(x_i)$ represents the value predicted by the model using a features vector $x_i$. To quantify the quality of the incorporated models’ training and testing predictions across the experimental features sets, we extract the following error quantifiers: (1) predictions coefficient of determination ($R^2$), (2) root mean squared (RMS) of the PEs, and (3) standard deviation of prediction residuals error wideness distances (${\delta }_{\mathrm{EWD}}$). $R^2$ is defined as $1-(R_{ss}/T_{ss})$, where $R_{ss}$ is the residual sum of squares, $T_{ss}$ is the total sum of squares, and $R^2$ indicates the quality of model training and testing processes, with values closer to “1” reflecting higher quality. The RMS of predictions errors is defined as $\mathrm{SQRT}(\frac{1}{n}*\sum _i^n{({\mathrm{PE}}_i)}^2)$, where $n$ is the total number of data points, and the result indicates the error in the predicted target, with values closer to “0” reflecting better predictions. To indicate the confinement of predictions around the 45 deg line, we measure ${\delta }_{\mathrm{EWD}}$, which is defined as $\mathrm{SQRT}(\frac{1}{n}*\sum _i^n{({\overline{\mathrm{PE}}}_i-{\mu }_{{\overline{\mathrm{PE}}}_i})}^2)$, where ${\overline{\mathrm{PE}}}_i=|{\mathrm{PE}}_i|/\sqrt{2}$ and ${\mu }_{{\overline{\mathrm{PE}}}_i}$ is the mean of ${\overline{\mathrm{PE}}}_i$ values. We use ${\overline{\mathrm{PE}}}_i$ instead of ${\mathrm{PE}}_i$, which has a mean close to zero and produces a standard deviation value close to the RMS value. The values of ${\delta }_{\mathrm{EWD}}$ closer to “0” reflect more confinement predictions around the 45 deg line.

5.2.

Modeling LEPB Using Density-Pixels Features

The training and testing LEPB prediction results ($\mu \mathrm{m}$) of the LightGBM model with the DENSITY_PIXELS features set are shown in Figs. 16(a) and 16(b), respectively. The error quantifiers are listed in Table 10, where the prediction RMS error values are $<1\mathrm{nm}$, with no observations of overfitting in the testing results, ${\delta }_{\mathrm{EWD}}$ results show confined predictions within 0.5 nm, and $R^2$ results show an overall good fitting of LEPB predictions to the input features set.

Fig. 16

LEPB predictions results using LightGBM and density-pixels features set: (a) LightGBM with density-pixels training and (b) LightGBM with density-pixels testing.

Table 10

LEPB PE quantifiers using LightGBM and density-pixels features set.

5.3.

Modeling LEPB Using Density-CCAS Features

The training and testing LEPB prediction results ($\mu \mathrm{m}$) of the LightGBM model with a DENSITY_CCAS features set are shown in Figs. 17(a) and 17(b), respectively, with close behavior correlation to the DENSITY_PIXELS features set. The error quantifiers are listed in Table 11, where the prediction RMS error values are $<1\mathrm{nm}$, with no observations of overfitting in the testing results, ${\delta }_{\mathrm{EWD}}$ results show confined predictions within 0.5 nm, and $R^2$ results show an overall good fitting of LEPB predictions to the input DENSITY_CCAS features set.

Fig. 17

LEPB predictions results using LightGBM and density-CCAS features set: (a) LightGBM with density-CCAS training and (b) LightGBM with density-CCAS testing.

Table 11

LEPB PE quantifiers using LightGBM and density-CCAS features set.

5.4.

Modeling LEPB Using GPS Features

The training and testing LEPB prediction results ($\mu \mathrm{m}$) of the LightGBM model with GPS features set are shown in Figs. 18(a) and 18(b), respectively, with very similar behavior to the DENSITY_PIXELS and DENSITY_CCAS features set. The error quantifiers are listed in Table 12, where the prediction RMS error values are $<1\mathrm{nm}$, with no observations of overfitting in the testing results, ${\delta }_{\mathrm{EWD}}$ results show confined predictions within 0.5 nm, and $R^2$ results show an overall good fitting of LEPB predictions to the input GPS features set.

Fig. 18

LEPB predictions results using LightGBM and GPS features set: (a) LightGBM with GPS training and (b) LightGBM with GPS testing.

Table 12

LEPB PE quantifiers using LightGBM and GPS features set.

5.5.

LightGBM Features Reduction

As we observe from the previous results, DENSITY_PIXELS, DENSITY_CCAS, and GPS features sets return very similar modeling results. Accordingly, from a representation efficiency point of view, all three approaches provide good fitting results with good prediction accuracy. However, are the features used by each representation approach as efficient as possible, or there is a redundancy that can be eliminated? LightGBM can produce a features importance list based on the number of splits in which each feature has been used. The more splits obtained, the more important the feature is. This list can be presented in descending order, where the most important features display at the top and the least important features at the bottom. Features with zero splits were not employed by the model in any splits and can be safely removed without affecting the modeling behavior or degrading the accuracy. Table 13 summarizes the feature reduction in each representation approach based on the LightGBM model feature importance by splits.

Table 13

Features reduction based on LightGBM splits.

DENSITY_PIXELS has no zero split features, indicating that for LEPB modeling through a regression-based ML modeling, a high-resolution density pixel is needed. Moreover, this information aligns with the fact that the information gain in each pixel is quite high, and each pixel is employed efficiently by the model, with no possible further feature reductions. DENSITY_CCAS has 13 features with zero splits ($-5\%$), aligning with the fact that CCAS has a type of information redundancy, as highlighted by Geng et al. in Ref. 24, since a dense number of sampling points implies circles and axes closer to each other, which accordingly implies redundancy. Figure 19 shows the training and testing LEPB prediction results ($\mu \mathrm{m}$), and Table 14 shows the PE quantifiers after applying features reduction to the DENSITY_CCAS features set. These results are almost identical to the results in Fig. 17 and Table 11 using the full set of DENSITY_CCAS features. GPS has 15 features with zero splits ($-7\%$) and all the reduced features are dimensional, which means these dimensional properties features were not important for LEPB modeling. Figure 20 shows the training and testing LEPB prediction results ($\mu \mathrm{m}$), and Table 15 shows the PE quantifiers after applying features reduction to the GPS features set. These results are almost identical to the results in Fig. 18 and Table 12 using the full set of GPS features.

Fig. 19

LEPB predictions results using LightGBM and Density-CCAS reduced features set: (a) LightGBM with CCAS reduced features training and (b) LightGBM with CCAS reduced features testing.

Table 14

LEPB predictions error quantifiers using LightGBM and density-CCAS reduced features set.

Fig. 20

LEPB predictions results using LightGBM and GPS reduced features set: (a) LightGBM with GPS reduced features training and (b) LightGBM with GPS reduced features testing.

Table 15

LEPB predictions error quantifiers using LightGBM and GPS reduced features set.

As we can observe from the results in Fig. 17, Table 11, Fig. 19, and Table 14 for DENSITY_CCAS representation, and Fig. 18, Table 12, Fig. 20, and Table 15 for GPS representation, there is almost no degradation in either modeling accuracy or error quantifiers after applying the features reduction.

Table 16 summarizes: (1) features counts, (2) features utilization referenced to density pixels (since it is the only features set that was not reduced), and (3) PE quantifiers of the LightGBM model. The error quantifiers are very similar in all the evaluation experiments, which indicates good model fitting to the LEPB predictions, and after features reduction, we can see that GPS features produced almost the same modeling quality (training: $\mathrm{RMS}=0.571\mathrm{nm}$, ${\delta }_{\mathrm{EWD}}=0.297\mathrm{nm}$, and $R^2\%=98.74\%$, and testing: $\mathrm{RMS}=0.643\mathrm{nm}$, ${\delta }_{\mathrm{EWD}}=0.344\mathrm{nm}$, and $R^2\%=98.40\%$) with $-22.22\%$ fewer features compared to the full features set of density pixels approach, and $-22.26\%$ fewer features compared to the full features set of the density CCAS approach.

Table 16

Summary of features counts, features utilization, and predictions error quantifiers.

5.6.

Further Discussion

In this study, we presented LEPB modeling using GPS direct one-to-one mapping geometrical features. However, GPS features are not limited only to the LEPB problem and can be incorporated further to model other systematic defects, such as pinching and bridging.