Differences

This shows you the differences between two versions of the page.

--- software:fitallb [2019/12/13 17:23] – estelle
+++ software:fitallb [2020/04/08 10:09] (current) – [Evaluating the results] matthias
@@ Line 36: / Line 36: @@
 ===== Usage =====
-==== FitAllB ====
+==== Prerequisites ====
-fitallb.py is the script that allow you to refine grains positions and orientations and to compute the strain tensor in the sample. To make it work, you need :
+''fitallb.py'' is the script that allows you to refine grain positions and orientations and to compute the strain tensor in the sample. To make it work, you need :
-  * the GrainSpotter file (.log)
+  * the GrainSpotter output file (''.log'')
-  * the original peaks list (.flt)
+  * the list of filtered peaks (''.flt'')
-  * the parameters of the phase (from imageD11, .prm)
+  * the ImageD11 parameter file of the phase (''.prm'')
-  * the structure file for your phase (.cif)
+  * the structure file for your phase (''.cif'')
-  * optionally, a res file, but i don't really know what that is so I don't use it;
+  * optionally, a //res// file, but i don't really know what that is so I don't use it;
-  * the input file (.inp)
+  * the input file (''.inp'')
+It furthermore requires //PolyXSim// and //Iminuit//. Both can be downloaded with //Anaconda// or is found on //GitHub//.
 The input file should have this look :
-<WRAP center round box 80%>
+<code>
 title 'title'
 log_file grains.log
@@ Line 84: / Line 86: @@
 #
 tol_grain 1e-2
-</WRAP>
+</code>
+==== Modifying the parameters ====
+Now, about the parameters of the refinement, these are the four lines starting with ''rej_'' and ''min_refl'' (maybe also the ''tol_grain'' line but it didn't do much in my tests..).
+''rej_ia'' is for internal angle, meaning the difference between the position of the measured peak and the position of the corresponding theoretical peak. In my data, I had a hard time decreasing it below 2° but I read that C. Langrand happened to have a maximum of 0.2° in internal angle misorientation.
-(I don't know how to add a whole file in the wiki or to formate the text so it don't look great...but you can get it from the edit page i guess. Or I can send the input file to you.)
+''rej_vol'' is linked to the intensity relativly to the volume of the grain (?). This means that the peaks have a too weak intensity. By default, it is set to 10 but I needed to increase it for my data. I succeeded in increasing up to 41 before //FitAllB// crashed (and it still rejected a lot of peaks because of intensity).
-In this input file, edit the part about the different related files and the one about your experiment conditions and then use :
+''rej_resmean'' and ''rej_resmedian'' define, how strong a single peak is allowed to be compared to the mean/median intensity of a certain grain. If you use the value ''10'' it means that any peak, which is more than 10 times higher intensity compared to the mean/median peak intensity of that grain, is rejected.
-<WRAP center round box 30%>
-fitallb.py -i inputfilename.inp
-</WRAP>
-In reaction, fitallb will run several fits in the teminal and if it succeed to refine your grain, it will end with a 'normal termination of fitallb'.
+''min_refl'' is the minimum number of peaks you need to validate a grain. As it is a refinement step, during each run the grains will loose the "bad" peaks. If they go below the min_refl threshold, they are rejected. Check your //log-file// before the refinement to see, how many peaks are usually there to make a grain.
-Then you have a new folder in the folder you are currently working in, named after your input file. In this new folder are all the output file fitallb created during the run. As I understand it now, the most usefull output files are the _final.gff and the _rej files.
-The _final.gff file contains the final results of the refinement, with the new grains positions, orientations, stress tensor,... This is the one you need to keep going with the data processing.
+==== Running the refinement ====
-The _rej file list all the peaks rejected during the refinement and tell you why they were rejeced. Most of the time it is a problem of intensity or internal angle (ia).
+Edit this input file to your purposes (modify file names, experimental conditions, crystallographic info, ...) and then type this to your command line:
+  fitallb.py -i inputfilename.inp
-Now, about the parameters of the refinement, these are the four lines begining by rej_ and the min_refl line (maybe also the tol_grain line but it didn't do much in my tests..).
+As a result, //FitAllB// will run several fits in the terminal and if it succeeds to refine your grains, it will finish with ''normal termination of fitallb''.
-rej_ia is for internal angle, meaning the difference between the position of the measured peak and the position of the theoretical peak corresponding. In my data, I have hard time decreasing it below 2° but I read that C. Langrand happend to have a maximum of 0.2° in internal angle misorientation.
+==== Evaluating the results ====
+Your current working directory should now contain a new folder named after your input file. In this new folder you can find several output files, which //FitAllB// created during the run. As I understand it now, the most usefull output files are the ''_final.gff'' and the ''_rej files''.
-rej_vol is linked to the intensity relativly to the volume of the grain (?) and reject peaks with a too weak intensity. By default, it was set at 10 but I need to increase it for my data. I succeeded to go up to 41 before fitallb crashed and it still rejecting a lot of peaks because of intensity.
+The ''_final.gff'' file contains the final results of the refinement, with the new positions, orientations, stress tensor, and so on of the grains. This is the one you need to keep going with the data processing.
+The ''_rej'' files list all the peaks rejected during the refinement and tell you why they were rejected. Most of the time it is a problem of intensity or internal angle (marked with ''ia'').
-I didn't used rej_mean and rej_median for now, I need first to know what it correspond to. I'll come back here when I find out..
+=== Reported crashs ===
+  * __maximum call reached__: Happens when //FitAllB// needs too much refinement loops to complete the calculation and is not powerfull enough to do it completly (?). Decrease your //rej_ia//, //rej_vol// or increase //min_refl// to fix it.
+  * __singular matrix__: Mean that fitallb does not have enough information on a grain to refine stress tensor. But same as before, decrease your //rej_ia//, //rej_vol// or increase //min_refl// to fix it. You can also solve this by removing the 'bad' grain in your input //log file//.
+  * __ValueError: invalid literal for int() with base 10: 'x,'__ (with x = any number): This happens when you use log-files, which were produced by merging several other log-files with //timelessGrainspotterMerge.py//. The problem can be solved when you search the log-file for two empty lines in a row and reduce them to one. The value of x gives you the grain number where to search for this in the log-file.
+  * Issues with the CIF file: Tip: If one (or multiple) of the following errors occur over and over again and you don't know what causes the error, there's a trick: Go to [[https://stokes.byu.edu/iso/isocif.php|ISOCIF]] and recreate your CIF file. If you have already a (not working) CIF file, this procedure takes only 2 minutes. In the ISOCIF output you have to modify the //symmetry_space_group_name_HM// according to the list provided [[https://github.com/FABLE-3DXRD/xfab/blob/master/xfab/sg.py|here]] and add this section:
-min_refl is the minimum number of peaks you need to validate a grain. As it is a refinement step, during each run the grains will loose the 'bad' peaks. If they go below the min_refl threshold, they are rejected.
+  loop_
+  _atom_type_symbol
+  _atom_type_description
+  'Mg' 'Mg'
+  'Fe' 'Fe'
+  'O' 'O'
-=== reported crashs ===
+  * If you don't want to use ISOCIF, here are some ways to deal with all the issues individually:
-  * I don't remember the exact error message, but it happen that fitallb say it can't read the input file. Most of the time it come from a typo in the called files name or because the file called is not in the current folder.
+    * CIF-file issue: __KeyError: u'No such item: _atom_type_symbol'__: Your CIF-file is required to have both ''_atom_type_symbol'' and ''_atom_site_type_symbol''. If one of them is missing, this error will pop up. Check other CIF-files to see how to implement these two things.
-  * "maximum call reached" : happen when fitallb need too much refinement loops to complete the calculation and is not powerfull enough to do it completly (?). Decrease your rej_ia, rej_vol or increase min_refl to fix it.
+    * CIF-file issue: __UnboundLocalError: local variable 'cf' referenced before assignment__: Again the CIF-file: Some old CIF-files contain ''_chemical_formula_sum'' twice (not sure why), one of them empty. Delete the empty one and try again.
-  * "singular matrix" : no idea about what it means...But same as before, decrease your rej_ia, rej_vol or increase min_refl to fix it.
+    * CIF-file issue: __KeyError: u'p4mnm'__ (or similar): It means that FitAllB cannot read the space group indicated in the CIF-file (wrong abbreviation). If you want to know, how your space group is abbreviated in a way that FitAllB can read it, check here: https://github.com/FABLE-3DXRD/xfab/blob/master/xfab/sg.py By the way: It is not case sensitive and ignores space.
+    * CIF-file issue: __UnboundLocalError: local variable 'unit_cell' referenced before assignment__: You probably have a typo in your crystal system (in your input file). I got rid of it, after I changed "orthorombic" to "orthorhombic".
+  * __TypeError: 'NoneType' object has no attribute '_getitem_'__ This error appears during the actual refinement of certain grains. Sometimes it can happen quite late in the refinement (on grain 300 or so), which makes it annoying, when the refinement is running over night and you only realize it next morning. There are two solutions for this: 1) Simply run //FitAllB// again and hope it works this time (sometimes it does actually). 2) If the error comes again and again at the very same grain, you have to delete that grain from the //log-file//. Normally, this would get you in trouble with the numbering, since the consecutive order of the grains gets destroyed, so you need to re-create the //log-file//. To avoid doing this every time when you run into this error, there is a trick: Simply don't delete all of the grain's information but just the majority. For example, keep one peak of the grain. This way, //FitAllB// will skip the grain in the refinement because one peak is lower than any number you usually put in the input file under //min_refl//. Run //FitAllB// again afterwards. Sometimes, you may have to delete several grains until it works.
+  * __LinAlgError: Singular matrix__ Use the same strategy as for the previous error in this list: Simply run //FitAllB// again and, if this doesn't work, make the software skip the grain by deleting most of the peaks for that particular grain.
 ==== Fitgloball ====
-fitgloball.py is a script linked to fitallb that do the same kind of things : refining grains. I used it only for refining cell parameters.
+''fitgloball.py'' is a script linked to //FitAllB// that does the same kind of things: refining grains. I used it only for refining cell parameters.
-It is working the same way as fitallb, with the same input file, just change d0 to 1 instead of 0 to ask the script to refine the cell parameters.
+It is working the same way as //FitAllB//, with the same input file, just change ''d0'' to 1 instead of 0 to ask the script to refine the cell parameters.
-A good advice to use it to refine cell parameters is to index grains with only the lowest tth (where it's easy for GrainSpotter to index peaks) and then use fitgloball on these 'perfect' grains. Then you can get back these refined cell parameters to run ID11 for the gve file and then keep going with GraainSpotter, fitallb,...
+A good advice to use it to refine cell parameters is to index grains with only the lowest tth (where it's easy for GrainSpotter to index peaks) and then use //Fitgloball// on these "perfect" grains. You can then go back to //ImageD11// to modify the parameters and calculate new (and hopefully better) G-vectors, then continue with //GrainSpotter//, //FitAllB//,...