====== GrainSpotter loop for a sample with multiple phases ======
Below is an example of a GrainSpotter loop for a multiphase sample. You can run it by typing
gsLoop-NPhases.sh > gsLoop-NPhases.log
The output of the loop will be saved in ''gsLoop-NPhases.log''.
===== Main idea =====
The options in this scripts are all in the top section of the file, which should be somewhat self-documented.
This script requires
* one g-vector file per phase, experimental g-vectors should be identical for all phases, but the list of computed theoretical g-vectors depends on the unit cell (see [[processing:complex-multi-phase|Indexing a complex dataset with many phases]]),
* some basic information on each phase (space group number, etc) and experiment wavelength,
* to update the GrainSpotter parameters (tthrange, etarange, domega, omegarange, etc)*
* to define parameters for each grain spotter loop:
* convergence criteria for each phase,
* number of random trials for each phase,
* how many times to shall we run GrainSpotter with those sets of parameters.
The number of times to run GrainSpotter, number of parameters to try, etc, is not limited. In some cases, with have run GrainSpotter up to 100 times, with 100000 random trials at each run. This is a test for 10 million configurations.
What it will do
- Create a folder for the indexing (Indexing-1, and then -2, etc if you request the loop to be run multiple times),
- Make a copy of the starting GVE file,
- Run the GrainSpotter iterations, saving a GrainSpotter output file for each iteration, and removing assigned experimental g-vectors from the GVE file (for all phases)
- Merge the indexed grains into a master ''merge-XXX-grains.log'' for each phase,
- Extract a list of Euler angles for those grains into ''merge-XXX-grains-euler.txt'' so you can plot the orientations of the indexed grains,
- Saving list of indexed g-vectors for each phase,
- Save the list of remaining g-vectors,
- Provide some indexing statistics using the script described in the section on how to [[evaluation:evaluate_indexing_statistics|evaluate your indexing performance]]
===== The loop =====
You wanted it?! Here is the source code for the loop above.
If you find that this script is useful for you, please cite one of the relevant [[http://timeless.texture.rocks/publications/ |TIMEleSS publications]] for which this script was slowly developped:
* One of the 2 Ledoux et al (2023) publications on olivine and wadsleyite,
* Gay et al (2023) on transformation microsctructures in pyrolite
* Upcoming (I hope) Gay et al publication on pyrolite deformation.
#!/bin/bash
#
# This is part of the TIMEleSS tools
# http://timeless.texture.rocks/
#
# Copyright (C) S. Merkel, Universite de Lille, France
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# Please cite one of the relevant TIMEleSS publications if you find that was of any use
#
# Number of phases
nphases=2
# Name of phases (used for comments)
phases=("Phase A" "Phase B")
# Short name of phases (for generating file names, avoid spaces and funny characters please)
# If your phase is BB, peaks will be in files peaks-BB.gve, grains in grains-BB.gve, and GrainSpotter input files in index-BB.in with additional numbers
phasesShort=("phA" "phB")
# Crystal symmetry codes, one per phase (used in merge grains)
# Codes can be found here http://multigrain.texture.rocks/doku.php?id=processing:compare-grain-orientations
crystalSymmetryCodes=(7 7)
# Space groups, one per phase (used in grainspotter)
spaceGroups=(245 212)
# Starting GVE files, one per phase
startGVEs=("phaseA.gve" "phaseB.gve")
# Wavelength (in Angstrom)
wavelength=0.34512
# Grain spotter parameters
# For each line:
# - How many times shall you run grainSpotter, parameters for phase 1, parameters for phase 2, parameters for phase 3
# - Parameter list for each phase: min_measuments min_completeness sigma_tth sigma_eta sigma_omega n_random
# You can add as many lines as you want. The indexing will be longer but the script will handle it.
gsPars=()
gsPars+=("5, 10 0.4 0.02 1 2 10000, 10 0.4 0.02 1 2 100") # iterations 0 to 4
gsPars+=("10, 10 0.4 0.02 2 4 10000, 10 0.4 0.02 2 4 100") # iterations 5 to 14
# gsPars+=("5, 10 0.4 0.02 1 2 10000, 10 0.4 0.02 1 2 100") # iterations 15 to 19
# Example
# gsPars+=("5, 10 0.4 0.02 1 2 10000, 10 0.4 0.02 1 2 100")
# adds 5 iterations with
# - for phase 1, 10 min_measuments, 0.4 min_completeness, 0.02 degrees sigma_tth, 1 degree sigma_eta and 2 degrees sigma_omega, 10000 random trials in GrainSpotter
# - for phase 2, 10 min_measuments, 0.4 min_completeness, 0.02 degrees sigma_tth, 1 degree sigma_eta and 2 degrees sigma_omega, 100 random trials in GrainSpotter
#
# gsPars+=("10, 10 0.4 0.02 2 4 10000, 10 0.4 0.02 2 4 100")
# adds 10 iterations with
# - for phase 1, 10 min_measuments, 0.4 min_completeness, 0.02 degrees sigma_tth, 2 degrees sigma_eta and 4 degrees sigma_omega, 10000 random trials in GrainSpotter
# - for phase 2, 10 min_measuments, 0.4 min_completeness, 0.02 degrees sigma_tth, 2 degrees sigma_eta and 4 degrees sigma_omega, 100 random trials in GrainSpotter
# Number of indexings (number of time to repeat the full thing to test for results consistency)
nIndexings=1
# Update parameters above as well as in
# - create_GS_init
# Location of executables
GrainSpotter=/usr/local/bin/GrainSpotter.0.90
clearGVE=/usr/bin/timelessClearGVEGrains
mergeGrains=/usr/bin/timelessGrainSpotterMerge
extractEuler=/usr/bin/timelessExtractEulerAngles
statistics=/usr/bin/timelessGSIndexingStatistics
#-----------------------------------------------------------------------------------------------
#
# Call to create GrainSpotter input
# 10 parameters
# - 1: name of GrainSpotter output file
# - 2: space group number
# - 3: gve file
# - 4: name of grain output file
# - 5: min_measurments
# - 6: min_completeness
# - 7: sigma_tth
# - 8: sigma_eta
# - 9: sigma_omega
# - 10: n_random
#
# Two theta ranges and other parameters for GrainSpotter are set by hand below.
# They are identical for all phases and mostly depend on the experimental settings.
# Please update according to your settings
#
# Other parameters are adjusted according to what was defined above in gsPars
#
#-----------------------------------------------------------------------------------------------
create_GS_init () {
cat > $1 <<- EOM
spacegroup $2 ! spacegroup [space group nr]
! dsrange 0 0.34 ! dsrange [min max], d-spacing range, multiple ranges can be specified
tthrange 3.5 6.95 ! tthrange [min max], multiple ranges can be specified
tthrange 7.15 9.95 ! tthrange [min max], multiple ranges can be specified
tthrange 10.1 12.2 ! tthrange [min max], multiple ranges can be specified
tthrange 12.35 14.0 ! tthrange [min max], multiple ranges can be specified
etarange 0 360 ! etarange [min max], multiple ranges can be specified
domega 0.5 ! domega [stepsize] in omega, degrees
omegarange -28 28 ! omegarange [min max] degrees, multiple ranges can be specified
filespecs $3 $4 ! filespecs [gvecsfile grainsfile]
cuts $5 $6 0.5 ! cuts [min_measuments min_completeness min_uniqueness]
eulerstep 5 ! eulerstep [stepsize] : angle step size in Euler space
uncertainties $7 $8 $9 ! uncertainties [sigma_tth sigma_eta sigma_omega] in degrees
nsigmas 2 ! nsigmas [Nsig] : maximal deviation in sigmas
! minfracg 1 ! stop search when minfracg (0..1) of the gvectors have been assigned to grains
! Nhkls_in_indexing 15 ! Nhkls_in_indexing [Nfamilies] : use first Nfamilies in indexing
random ${10} ! use randomly chosen orientation seeds #trials
! positionfit ! fit the position of the grain
! genhkl ! generate list of hkl's based on space group and cell parameters in gve file
EOM
}
#-----------------------------------------------------------------------------------------------
#
# NOTHING SHOULD BE CHANGED BELOW
#
#-----------------------------------------------------------------------------------------------
#-----------------------------------------------------------------------------------------------
#
# Actual indexing loop
# Runs tens of GrainSpotter iterations with decreasing tolerances
# Runs in the current folder
#
#-----------------------------------------------------------------------------------------------
indexing_loop () {
# Copy GVE file for each phase
echo "#"
echo "# Copy of starting GVE files into current working folder"
echo "#"
for (( i=0; i<$nphases; i++ ))
do
if [ ! -f "$home/${startGVEs[$i]}" ]; then
echo "Error: starting GVE file ${startGVEs[$i]} does not exist in folder $home"
echo "Can not go any further"
exit
fi
cp -f $home/${startGVEs[$i]} peaks-${phasesShort[$i]}.gve
cp -f $home/${startGVEs[$i]} peaks-ori-${phasesShort[$i]}.gve
echo "Saved ${startGVEs[$i]} as peaks-${phasesShort[$i]}.gve and peaks-ori-${phasesShort[$i]}.gve"
done
echo "Done"
# Loop on GrainSpotter indexings
i=-1
for gsPar in "${gsPars[@]}"
do
# Set comma as delimiter
IFS=','
# Split parameter information
read -a pars <<< "$gsPar"
# Number of GrainSpotter loops for this set of parameters
nloops=${pars[0]}
# Extracting GrainSpotter parameters for each phase
parphases=()
for (( j=0; j<$nphases; j++ ))
do
((k = j + 1))
parphases[$j]=${pars[$k]}
done
# Run loop of GrainSpotter indexings for these parameters
for (( loop=0; loop<$nloops; loop++ ))
do
((i = i + 1))
echo "#"
echo "# Running GS iteration $i. GS parameters in index-XXX-$i.ini, grains saved in grains-XXX-$i.log for each phase"
echo "#"
echo "#"
for (( j=0; j<$nphases; j++ ))
do
echo "#"
echo "# GS iteration $i. Parameters for ${phases[$j]}: ${parphases[$j]}"
echo "#"
# Splitting indexing parameters as array, otherwise it does not work
IFS=' '
read -a pars <<< "${parphases[$j]}"
# Creating GrainSpotter input file and running the indexing
create_GS_init index-${phasesShort[$j]}-$i.ini ${spaceGroups[$j]} peaks-${phasesShort[$j]}.gve grains-${phasesShort[$j]}-$i.log ${pars[@]}
$GrainSpotter index-${phasesShort[$j]}-$i.ini
lastGSInput[$j]=index-${phasesShort[$j]}-$i.ini
echo "#"
echo "# Done with GS indexation on ${phases[$j]}."
echo "# Indexing saved in grains-${phasesShort[$j]}-$i.log"
echo "#"
grainlists[$j]="${grainlists[$j]} grains-${phasesShort[$j]}-$i.log"
echo "#"
echo "# Clear indexed peaks from GVE lists"
echo "#"
# Removed used peaks from gve lists
for (( k=0; k<$nphases; k++ ))
do
$clearGVE grains-${phasesShort[$j]}-$i.log peaks-${phasesShort[$k]}.gve tmp.gve
mv -f tmp.gve peaks-${phasesShort[$k]}.gve
echo "#"
echo "# New peak list for ${phases[$k]} saved in peaks-${phasesShort[$k]}.gve"
echo "#"
done
echo "#"
echo "# Done with GS iteration $i on ${phases[$j]}"
echo "#"
done
done
done
}
#-----------------------------------------------------------------------------------------------
#
# Indexing loop, runs tens of GrainSpotter iterations with decreasing tolerances
# Runs in the current folder
#
#-----------------------------------------------------------------------------------------------
# Saving home directory
home=`pwd`
# Global parameter. Used to pass list of files between functions
grainlists=()
lastGSInput=()
for (( i=0; i<$nphases; i++ ))
do
grainlists[$i]=""
lastGSInput[$i]=""
done
# Make some tests to make sure all parameters have been set
error=false
for (( i=0; i<$nphases; i++ ))
do
if [ -z "${phases[$i]}" ]; then
((k = i + 1))
echo "Name of phase $k is blank. This should not happen. Check definition of \$phases variable.";
error=true
fi
if [ -z "${phasesShort[$i]}" ]; then
((k = i + 1))
echo "Short name of phase $k is blank. This should not happen. Check definition of \$phasesShort variable.";
error=true
fi
if [ -z "${crystalSymmetryCodes[$i]}" ]; then
((k = i + 1))
echo "Crystal symmetry code for phase $k is blank. This should not happen. Check definition of \$crystalSymmetryCodes variable.";
error=true
fi
if [ -z "${spaceGroups[$i]}" ]; then
((k = i + 1))
echo "Space group for phase $k is blank. This should not happen. Check definition of \$spaceGroups variable.";
error=true
fi
if [ -z "${startGVEs[$i]}" ]; then
((k = i + 1))
echo "Name of file for GVE vectors for phase $k is blank. This should not happen. Check definition of \$startGVEs variable.";
error=true
fi
done
ngs=-1
for gsPar in "${gsPars[@]}"
do
# Set comma as delimiter
IFS=','
# Split parameter information
read -a pars <<< "$gsPar"
# Number of GrainSpotter loops for this set of parameters
nloops=${pars[0]}
((ngs = ngs + nloops))
# Extracting GrainSpotter parameters for each phase
for (( j=0; j<$nphases; j++ ))
do
((k = j + 1))
if [ -z "${pars[$k]}" ]; then
echo "No parameter for phase $k at line $gsPar"
error=true
fi
done
done
if [ "$error" = true ]; then
echo ""
echo "There were errors while parsing the script parameters. See messages above."
echo ""
exit
fi
echo "# "
echo "# "
echo "# Multiphase multigrain indexing"
echo "# "
echo "# This script will run $nIndexings attemp(s) of an indexing with $nphases phases, with $ngs GrainSpotter iterations for each phase."
echo "# Phase list: ${phases[@]}"
echo "# Starting GVE-vectors saved in ${startGVEs[@]}"
echo "# Parameters for GrainSpotter indexings"
for gsPar in "${gsPars[@]}"
do
# Set comma as delimiter
IFS=','
# Split parameter information
read -a pars <<< "$gsPar"
# Number of GrainSpotter loops for this set of parameters
nloops=${pars[0]}
echo "# - $nloops iteration(s) with"
# Extracting GrainSpotter parameters for each phase
for (( j=0; j<$nphases; j++ ))
do
((k = j + 1))
echo "# * ${pars[$k]} for ${phases[$j]}"
done
done
echo "#"
echo "#"
for (( loopIndexing=1; loopIndexing<=$nIndexings; loopIndexing++ ))
do
echo "#----------------------------------------------------------------------------------------"
echo "#"
echo "# Full indexing loop number $loopIndexing"
echo "# Data saved in folder: Indexing-$loopIndexing"
echo "#"
echo "#----------------------------------------------------------------------------------------"
echo ""
folder="Indexing-$loopIndexing"
\rm -rf $folder
mkdir $folder
cd $folder
# Clear list of indexed grains
for (( i=0; i<$nphases; i++ ))
do
grainlists[$i]=""
done
# Start indexing loop
indexing_loop
echo ""
echo "#---------------------------------------"
echo "# Indexing loop number $loopIndexing is finished"
echo "#"
# Merge grains for each phase
mergeGrainFiles=()
originalGVEFiles=()
for (( i=0; i<$nphases; i++ ))
do
originalGVEFiles[$i]="peaks-ori-${phasesShort[$i]}.gve"
echo ""
echo "#---------------------------------------"
echo "# Merging grain information for ${phases[$i]} into files starting with merge-${phasesShort[$i]}, i.e. merge-${phasesShort[$i]}-grains.log, etc"
echo "#"
$mergeGrains ${grainlists[$i]} -c "${crystalSymmetryCodes[$i]}" -o "merge-${phasesShort[$i]}"
mergeGrainFiles[$i]="merge-${phasesShort[$i]}-grains.log"
echo ""
echo "#---------------------------------------"
echo "# Extracting Euler angles for ${phases[$i]} into merge-${phasesShort[$i]}-grains-euler.txt"
echo "#"
$extractEuler -o "merge-${phasesShort[$i]}-grains-euler.txt" "merge-${phasesShort[$i]}-grains.log"
echo ""
echo "#---------------------------------------"
echo "# Saving list of indexed g-vectors for ${phases[$i]} into gve-Indexed-${phasesShort[$i]}.gve"
echo "# List of remaining g-vectors for ${phases[$i]} moved to gve-NotIndexed-${phasesShort[$i]}.gve"
echo "#"
$clearGVE -k gve-Indexed-${phasesShort[$i]}.gve merge-${phasesShort[$i]}-grains.log peaks-ori-${phasesShort[$i]}.gve tmp.gve
rm tmp.gve
mv peaks-${phasesShort[$i]}.gve gve-NotIndexed-${phasesShort[$i]}.gve
done
echo ""
echo "#---------------------------------------"
echo "# More indexing statistics"
echo "#"
$statistics -w $wavelength -i ${lastGSInput[@]} -l ${mergeGrainFiles[@]} -g ${originalGVEFiles[@]}
cd $home
echo ""
echo ""
echo "#----------------------------------------------------------------------------------------"
echo "#"
echo "# Done with indexing loop number $loopIndexing"
echo "# Data saved in folder: Indexing-$loopIndexing"
echo "#"
echo "#----------------------------------------------------------------------------------------"
echo ""
echo ""
done
exit