Package Architecture

PyExtal is composed of two main components:

  1. Bloch wave simulation engine

  2. Interface for optimization and analysis

Bloch Wave Simulation Engine

Bloch Wave Simulation Engine is responsible for simulating diffraction intensity of a crystal structure. It is adapted from the code described in Electron Microdiffraction by Zuo and Spence and ships as a compiled extension module, pyextal.blochwave, called from Python.

The engine parses the input, computes structure factors and selects the strongly excited beams. The diagonalization of the structure matrix runs with numpy.linalg.eig, with incident directions distributed over a thread pool (see Simulation Engine).

The engine has three main subroutines:

  • bloch_parse:
    Reads the input file and parses the parameters for crystal structure, experiment and simulation conditions. This must be called first before any other subroutines as it initializes the parameters.

  • bloch_run/tilt_run: Performs the Bloch wave simulation. Based on the input parameters, bloch_run will determine a set of incident directions and solve for eigenvalue and eigenvector for each direction. The tilt_run subroutine accepts a set of predefined incident directions and solve for eigenvalue and eigenvector for each direction, which is useful for refinement where we need to specficy the incident directions.

  • LARBED: Performs the intensity calculation for a given set of eigenvectors and eigenvalues. This allows us to calculate the diffraction intensity of different thickness without redoing the expensive calculation of eigenvectors and eigenvalues. The calculation is described by equ 5.17 in Advanced electron microscopy by Zuo and Spence.

  • cbedp: This is a legacy subroutine that has the same function as LARBED. It is still used for some part of the code, but it is subject to be removed in the future.

Interface for Optimization and Analysis

The interface for optimization and analysis is implemented in Python in a object-oriented fashion. It is designed to be user-friendly and easy to use. The interface is composed of several modules, each responsible for a specific task. The main modules are:

dinfo:

This is a set of classes that contains the information of the diffraction pattern, including the pattern itself, initial esitmates of sample thickness, sample orientation, calibration of the pattern pixel size and path to the input file containing crystal and simulation parameters. For LARBED pattern, it also accepts the pre-calculated pixel-wise variance of intensity. It also contains methods for managing structure factors, so users can access the refined structure factors.

ROI:

This is a set of classes that contains the information of the region of interest (ROI) in the diffraction pattern. It is used to define the area of the diffraction pattern that will be used for optimization. This class also manages the connection between the simulation and the experimental data.

optimize:

This module contains two classes: Coarse and Fine.

  • Coarse Refinement: The Coarse refinement refines the parameters by comparing the experimental data with the simulated data with cross-correlation. The simulation is done with a grid sparser than the experimental data and upsampled to reduce the computation time. A set of predefine optimzation traget functions are defined.

    • thickness: thickness of the sample

    • gl: scaling factor for the diffraction pattern, equivalent to camera length for CBED and tilting step size for LARBED

    • DWF: Debye-Waller factor, which is used to account for the thermal vibration of atoms in the crystal structure. The DWF is defined as \(exp(-Bsin^2(\theta)/\lambda^2)\), where B is the Debye-Waller factor, theta is the scattering angle and lambda is the wavelength of the incident beam. The DWF is used to correct the intensity of the diffraction spots for thermal motion due to inelastic scattering.

    • cell: unit cell parameters, including the lattice parameters and the angles between the axes. The unit cell parameters are used to define the crystal structure and are used to calculate the diffraction pattern.

    • orientation: orientation of the crystal structure with respect to the incident beam. Refining this doesn’t require extra calculation, instead, a field of view larger than the roi is simulated and the orientation is refined by finding the position that gives best match with the experimental data. This is often in tandem with other parameters, such as thickness and gl.

  • Fine Refinement: fine refinement simulated the intensity pixel by pixel. Several different goodness-of-fit (GOF) metric are defined in the gof module. Optimization is performed using the scipy.optimize module and the default optimizer is the Nelder-Mead simplex algorithm. The fine refinement is solely for structure factor refinement. For each iteration, the structure factors are adjusted and the simulation is performed. Subsequently, the scaling factor(gl), thickness and orientation are also refined to give the best fit based on the defined GOF metric. The fine refinement will stop once the stopping criteria is met.

larbed:

This subpackage covers the step before refinement: turning a raw EMPAD 4D-STEM acquisition into the rocking curves the rest of the package consumes. LARBEDAnalysis reads the raw file, indexes the reflections on a reciprocal-space grid built from two (or four, including HOLZ) basis vectors, and integrates each reflection through a circular or annular aperture, propagating the counting variance. LARBEDCalibration measures the detector MTF and the tilt-scan step length from a reference scan. The result is handed straight to a dinfo object with to_dinfo, or written to the shared HDF5 file with save_h5. This was merged in from the standalone PyLARBED package; see LARBED Extraction.

adp:

This module holds the temperature-factor bookkeeping the engine needs but never exposed. The Bloch engine stores a full anisotropic tensor beta per atom (blochwave.cryst.atpar rows 3…8) and applies exp(-h^T beta h) to the elastic scattering; blochwave.cryst.dw is the derived equivalent B_eq that only the absorptive form factor reads. adp converts between isotropic B, Uij and beta, writes both halves consistently with set_beta, and – the part that matters for refinement – derives each atom’s site symmetry from the space group so allowed_beta_basis can say which components of beta are actually free. DWFModel wraps that into the parameter vector CoarseOptimize.optimizeDWF refines, in units of Angstrom^2 of B. See Coarse Refinement.

gof:

This module contains several classes that define different goodness-of-fit (GOF) metrics. The GOF metrics are used to evaluate the quality of the fit between the experimental data and the simulated data. Users can define their own GOF metrics by inheriting from the base class BaseGOF. BaseGOF is an abstract base class that defines the interface for the GOF metrics.

Basic Workflow

  1. Prepare dataset:

    • CBED: Energy-filtered CBED patterns should be indexed and deconvolved to remove the MTF of detector. This package comes with a Lucy-Richardson deconvolution implementation that can be used to deconvolve the CBED patterns. The deconvolved patterns should be saved in a directory.

    • LARBED: LARBED patterns should be extracted from the raw data, together with the variance of the intensity for each pixel. The larbed subpackage does this and can hand the result directly to step 3, skipping the intermediate files.

  2. define input .dat file: .dat file is a configuration file that contains the crystal structure, simulation parameters and diffraction gemometry.

  3. User should first create a dinfo object with the input file. CBED/LARBED pattern also needs to be inputed.

  4. Then, a ROI object should be created to define the region of interest in the diffraction pattern.

  5. After that, a CoarseOptimization object can be created to perform the coarse refinement.

  6. Subsequently, a FineOptimization object can be created to perform the fine refinement. The final results can be accessed from the dinfo object.

detail explaination of the workflow