Coarse Refinement (Correlation based refinement)

In order to reduce calculation time, some parameters can be optimized with correlation metric, especially for those parameters that are sensitive to feature positions (e.g. holz line, rocking curve spacing etc.). This is done by simulating intensity of a predefined range of tilt wider than experiment field of view in reciprocal space using given parameter. The simulation can be done with a grid coarser than experiment data to reduce computation time. Experiment data is then used as a template to find the tilt position with highest correlation coefficient. This identify the best incident beam direction. In pyExtal, we applied the features.match_template function in scikit-image library for the matching. This process is defined as the target function, which is feed into scipy.optimize.minimize function with method='brent' option for optimization. All the optimization is done in the display space as described in ROI

To perform the coarse refinement, you need to set up a CoarseRefine object with the following parameters:

from pyextal.optimize import CoarseOptimize
coarse = CoarseOptimize(datpath='path/to/dat', dinfo=dinfo, roi=roi)

image preprocessing

Sometimes it is also advisable to use some image filtering techniques to enhance the features of interest. We provided a argument that allows user to pass in any kind of image preprocessing function with following singature to enhance the features.

def preprocess(image: np.ndarray, sim=False) -> np.ndarray:
    """
    Preprocess the input image to enhance features for correlation matching. 
    Simulation and experiment patterns may require different preprocessing techniques as the range of intensity is different. 
    Normalizaltion might be needed for thresholding or filtering.
    
    Parameters:
    - image: Input image as a numpy array.
    - sim: Boolean indicating if the image is a simulation pattern (default is False).
    
    Returns:
    - Preprocessed image as a numpy array.
    """
    if sim:
        res = some_simulation_preprocessing_function(image)
    else:
        res = some_experiment_preprocessing_function(image)
    return res

This provides great flexibility for users to apply their own image processing techniques. Some predefined preprocessing functions will be shipped with pyExtal in the future.

geometry refinement

Geometry parmeters inlcude the incident beam direction, step size, and sample thicknss. In pyExtal, step size and sample thickness can only be refined separately and simultaneously with sample thickness. This is due to the fact that both step size and thickness affect the features in the diffraction pattern, so doing them together will lead to convergence issues.

  • CoarseRefine.optimizeOrientationThickness: This function optimizes the target function using thickness as input parameter.

  • CoarseRefine.optimizeOrientationGL: This function optimizes the target function using step size (gl) as input parameter.

HV calibration

In CBED/LARBED experiments, the accceleration voltage (HV) is a critical parameter that affects the diffraction pattern. Similar to geometry parameter, a target function taking hV as input can be optimized. The CoarseRefine.optimizeHV function is used to optimize the HV parameter.

Debye-Waller factor

The Debye-Waller factor describes the thermal vibration of the atoms. optimizeDWF() refines it against the same cross-correlation the other coarse targets use, re-fitting thickness and orientation inside every evaluation so the comparison stays fair.

coarse.optimizeDWF()                       # one isotropic B per atom
coarse.optimizeDWF(aniso=True)             # symmetry-allowed anisotropic tensor
coarse.optimizeDWF(group='type')           # tie atoms of the same element together

Anisotropic refinement

The engine has always applied a full anisotropic tensor – exp(-h^T beta h) with beta in blochwave.cryst.atpar rows 3…8 – so aniso=True refines what the simulation was already using rather than adding a model. What it does not do is refine all six components blindly: pyextal.adp derives each site’s symmetry from the space group and refines only the directions that symmetry leaves free.

For SrTiO3 that is one parameter each on Sr (1a) and Ti (1b) and two on O (3c, site symmetry 4/mmm) – four in total, not eighteen. Every coefficient is expressed in Angstrom^2 of equivalent B, and the coefficients of one site sum to its B_eq, so the bounds and starting values mean the same thing they did for the isotropic refinement.

Refined tensors can be written back into a .dat with pyextal.adp.atom_line(), in a file whose crystal line declares dw = bij.

Cost

Each evaluation costs a full re-diagonalization plus the inner thickness fit, so the practical question is how many are needed and what each one costs. Two settings dominate:

window

Half-width of the correlation search box, in pixels, held around the known peak. Off by default. The full search re-examines every template offset in the simulation – on the SrTiO3 example a 459x459 space, 56% of the cost of an evaluation against 15% for bloch_run – and window=24 cuts an evaluation from 56 s to 25 s.

It is not free, which is why it is opt-in. The inner thickness fit is not bracketed, so one evaluation can walk the thickness by 20 Angstrom, far enough for the best template offset to leave a small box; on the SrTiO3 example that moved the first evaluation’s score from 0.31104 to 0.31362. The window widens itself back to a full search when the peak approaches an edge the full search would have seen past, which covers drift up to about the window size, and optimizeDWF re-checks the final peak with a full search and warns if it moved further. Use it once the orientation has converged.

constant['coarse_DWF']['options']

ftol and xtol, in the units of the objective and of B. These are the only real control over how many evaluations the search spends. The options shipped before were Nelder-Mead spellings (fatol, xatol, adaptive) handed to Powell, which discards them – so the search silently ran on scipy’s defaults, resolving B to 1e-4 Angstrom^2 against a metric that cannot see a change of 0.14 Angstrom^2. They are now ftol=1e-3, xtol=1e-2, set from what the correlation actually resolves.

Note

Scaling Powell’s direc looks like it should shorten each line search, and does not. With bounds given, _linesearch_powell searches the whole feasible interval along each direction and measures its tolerance in the step variable, so a smaller direc converges tighter and costs more evaluations.

Warning

This target can run to its bounds. On the SrTiO3 LARBED example 1 - corr falls monotonically as B(Sr) goes 0.62 to 2.0, and all three atoms finish pinned against the 2.0 Angstrom^2 bound from starting values of 0.62 / 0.44 / 0.80 – values that are physically wrong for SrTiO3 at room temperature. The cross-correlation prefers more thermal damping without ever turning over, so the answer is set by constant['coarse_DWF']['boundary'] rather than by the data. optimizeDWF prints a warning when a parameter lands on a bound.

The local sensitivity is small as well: a 23% change in B(Sr) moves the objective by 5e-5. Between the two, treat a coarse DWF as a starting estimate at best, and refine temperature factors against the chi-squared metric in Fine Refientment (Intensity based Refinement) or the JAX stage.

unit cell refinement

Unit cell parameters can also be refined using the coarse refinement method. The CoarseRefine.optimizeUnitCell function is used to optimize the unit cell parameters. This function takes the target function and optimizes it with respect to the unit cell parameters. As we have 6 unit cell parameters, the optimization is done with respect to all 6 parameters simultaneously using method='Nelder-Mead' option in scipy.optimize.minimize.