NvJ has a LACS (Linear analysis of Chemical Shifts) plot. NMRFx should have that too.
Generation of NOESY peaks based on the RNA secondary structure will be much faster if the code is moved from Python to Java
The idpeak tool uses chemical shifts from the atoms. The user should be able to specify whether this is the assigned shift or the reference shift.
When using the nd.create() scripting command to create a new dataset the user should be able to get the sizes from an existing one.
Arrangements of spectra in RunAboutX is controlled by parameters loaded from the runabout.yaml file. Users should be able to load their own arrangements from a file.
Various experiments on Bruker systems (NOESY, ARTSY) are done where some parameter is arrayed before the phase and time increments. The appropriate Acq Mode for these is “a2,p1,d1”. This should be an option in the Acq Mode combo box so it is easy to select
Add the ability to export 1D and 2D files to .csv files so they can be analyzed in external software
The atom assignments place on peak labels are off by one residue
Loading NMR-STAR files with chemical components (ligands) may fail
Remove this python code that isn't used
Optimize GRINS NUS fitting and add operations for preparing (phase, anodization, zero-filling) prior to NUS operations
Bruker files have an SF01 in each acquXs file and have SF01, SF02, SFO3 etc. in acqus file. Generally the SFO1 in the indirect dimension acquXs files is correct, and the same as the corresponding value in acqus, but not always. We should get SFO from the correct value in acqus
Generating structures from NEF files should be improved and the code will be clearer if this is moved from Python to Java
Use a bipartite matcher for post-processing secondary structure prediction
Molecules can be displayed on canvas automatically if present, but this is inappropriate for large molecules.
RDC calculations currently use a hard coded distance for H-N and H-C atoms. If they the atoms are not directly bonded the distance should be calculated from the structure
When checking for lower probability base pairs only add them if there is a neighboring basepair
NMRFx tries to setup apodization consistent with what is in the data file, but if its not present (as in SpinIt files) it will fail with an NPE
Add code from the NMRViewJ peak network matching tool for performing bipartite matching optimized genetic algorithm for assigning peaks
Phases can be present in the original dataset (in procs for Bruker etc.).
We should initialize the process script phases to those values. This may entail some conversion as different vendors have different phasing conventions.
A section should be added to the Preferences to control what should happen when opening an FID.
Including:,
1) Whether to auto process (current default for 1D/2D)
2) Whether to use any saved (and non-zero) phases, or whether to do an autophase
3) Possibly whether to use other parameters (apodization, linear prediction, zero filling) that are in the file.
4) Whether to open and display molecule file if present in fid directory
If you’ve peak picked an 1D spectrum and shown the multiplet tool you’ll have a mode set where any addition of regions will result in peak picking those regions. If you clear a peak analysis and do a region add it will still do the peak picking. This mode should be cleared when the analysis is cleared.
When opening FID files users are prompted for a name to save the processed data in. We should skip that when opening a .dx file and just use an output name (for FID file.dx) of file.nv.
Benchtop spectra come in with intensities around 1e6 . Should set a scale value to put them in a more friendly range.
Peaks can be picked by dragging the cursor (in Peak mode) across the 1D spectrum. Doing this over a region with existing peaks can delete other peaks. Also, the limit on how low the threshold can go is too high, preventing the user from picking small peaks.
Molecules generated from SMILES string don’t get a correct entity number which creates problems when reading/writing STAR files
The TRACT tool currently requires that the x axis data come from a column named etime. We should add more possibilities (value and delay)
Users should be able to deactivate individual rows in the scanner table so the data from that row is not displayed or used in the fitting in the plot tool.
If you have the scanner table opened it would be more convenient to adjust the levels for individual rows if the normal up/down scale and autoscale buttons only acted on selected rows (or all rows if none selected)
Spectra with journal annotations aren’t properly written or read from .yaml files used for saving the spectrum display
Before processing the user is prompted for a filename. At present a full file dialog is displayed.
Change this to just a smaller prompt for the filename.
Clicking the File... button could be used to get full file dialog
Add a version of the scanners plot tool that has specific features for diffusion analysis
Add tool to calculate score of predicted spectra with experimental spectra and new layout tool to read a layout from .yaml file
NMRFx Analyst can predict RNA secondary structures using a deep learning model, but the model is not distributed with NMRFx (to minimize size of installer). We should add the ability to fetch the model from nmrfx.org
Experiments like HCCH Tocsy are often aliased in the carbon dimension. We should support automatic unaliasing of these based on expected chemical shift ranges.
Add scoring of RNA predictions against experimental data and ability to define spectrum layouts.
Add GUI commands for adjusting shift and coupling constants in quantum prediction data (from GISSMO)
When doing “Load from Dataset in the Scanner table the code should check to see if the dataset has associated values (for example, ones loaded from Bruker vdlist or vclist or Varian array parameters). A column should be added with those values so they can be used in relaxation or diffusion analysis.
Do volume calculation when picking 1D peaks, allow selecting peaks when label is empty, default to Atom Label mode in Analyzer
Improve the display of integral values on 1D spectra. By default show the integral value as horizontal text at bottom of integral region. Illustrate the range of region with a horizontal bar above the text. Dynamically offset text and range bar vertically to prevent overlap with adjacent values.
Solvent names can have different cases. We should put and get them in hash map with lower case names
Use pascal triangle sum to calculate norm value for multiplet volume
Picking in box mode picks entire window, not just the region within crosshair box
When loading a spectrum with a valid lvl value in .par file autoscale should not be done
When you load a new dataset some reference parameters aren’t cleared from previous dataset
Lvl and offset behaviour is not correct
When converting a 2D flat molecule to 3D it would be good to retain the original 2D (flat) coordinates for display on spectrum and 2D viewer.
Add undo/redo to the edit menu so actions like compress, Degas, mirror etc. can be undone
Improve ability to switch chart display between overlay and grid and have scanner table work.
Molecules created via SMILE strings aren’t stored in project so can’t be used
Exceptions are thrown during structure calculation in the GUI because the collection storing active structures is accessed by both the structure calculation and GUI threads
Move from Java 17 to Java 21
Add a filter entry to the Vendor Pars table view so that only parameters whose name matches the filter text will be displayed.
Extracted vectors don’t display until mode is changed from Contours to Trace
Move the slice checkbox back to the spectrum status bar (from the attributes panel) so it is more convenient for users. Leave the slice controls (colors etc.) in the Slice pane of attributes panel
Switch from tensorflow snapshot to 1.0rc1
Typing a number into the cursor position text boxes currently can be used to set crosshair positions. In cursor Selector mode it should be possible to set the plot limits in the same way
Add more key bindings (based on ones in NMRViewJ)
The peak attributes section should have controls for the peak font size and for the range of planes to use for drawing peaks
Add buttons for FID/Dataset (its not obvious that double-clicking opens datasets, and that control-clicking opens FIDs. Allow sorting by columns. Add column for text (title). Default dataset should be the most recently modified one. Position browser at top left of window and set to stay on top.
Allow loading RNA secondary structure from files and allow selecting from a list of predicted and loaded secondary structures. Also show map of base pair predictions
Add operations to convert a real vector to complex and to zero the imaginary or real parts of complex vectors.
Currently we skip some angles that could be constrained by suite parameters
Use just the atom names, not peak ids, for ResonanceSource hash code, ,so that resonance sources from multiple peak lists have same hash code
If the dataset dimension that is arrayed (as in relaxation experiments) isn’t set, detect it by finding the smallest dimension
After setting up attributes like contour levels, positive and negative drawing status, and colors, the user should be able to set those attributes on to the dataset and then save the parameter file. This allows the user to reload the dataset in another project and have appropriate attributes. This is a feature that was in NMRViewJ.
Add a text field, bound to the slider, for setting the aspect ratio attribute.
getActive should activate a molecule if none are currently active
Loading XEASY peak files is useful for testing
If a structure is read with multiple models it should be possible to see all the models (and select which ones are shown) in the viewer.
Add peakid tool (like in NvJ) and improve peak id tools in noe table
Various small fixes for structure calculation including fix for superimposing after calcs, dropping ambiguous restraints inappropriately (if in fixed group
Fixes for distance constraint tables and viewer
Some NEF files have a chain code “.” (meaning there is no set value) in the chemical shift assignment save frame. NMRFx throws an exception when trying to open these.
GRINS requires appropriately phased data. It has been possible to specify phases for the indirect dimensions before now, but there wasn't support for negatePairs or negateImag, which often has to be done to the TD data before FT and phasing.
Improve display of clusters and sequence in RunAbout and add other general RunAbout improvements
NMRFx incorporates git for project management, but needs a GUI for interacting with it
Creating and destroying top-level windows eventually exhausts video memory. This can happen after even 10 times depending on how large the windows are and the screen resolution. When stages are removed the canvas objects are not properly garbage collected so video memory used by the canvas is not released.
Allow adding plugins whose controls are in the right side tool pane
Add tools to fetch search BMRB, fetch entries and upload depositions
Allow setting chart position (row/column) and span in grids
Tensorflow Java 1.0-SNAPSHOT supports Mac Silicon processors so by updating to this release we won’t have to build the Mac Silicon (ARM) version of tensorflow ourself, and will get support for tensorflow 2.16.1
Add menu entry and code to fetch NMR-STAR files from the BMRB.
Add method for RING to request peaks to be displayed in NMRFx
Add more menu options for controlling structure refinement with annealing
You can currently only do Refine or Calc Structure (from Molecule Viewer Calculate menu) once. A second try will corrupt the structure if the energy is 0.0
Regression introduced in recent commit
Currently methods and constructors for finding and creating NMRData classes (BrukerData etc.) receive the file path as a string. It would be better to pass File objects.
Allow selecting atoms and adding constraints in 3D mol viewer. This will be useful for testing, demonstration and use with small molecules.
Features
Currently only unassigned peaks are extracted as constraints. Should extract assigned peaks also.
Spinit uses NUS files that have entries for each phase increment. These should be supported
Updates to spectrum adjuster, parameter saving and add more methods
Add fitting of titration data to a model with two independent sites
Reading annotation polylines is not currently supported, but they are used in RunAbout. Should add support for them.
Allow peak pick of 3D datasets where the 1st and 3rd dimensions are frequency dimensions, but the 2nd is an arrayed experiment. This happens often with relaxation experiments.
Add an operation for tilting j-resolved spectra
Adding region columns to the scanner table is currently done with the Regions menu after positioning crosshairs. It would be easier if dragging the cursor in Region mode would add these regions if a scanner table is present. Currently the regions are added to dataset, but don’t appear as a table column
The REGION operation is in the Baseline Correction operation group. That means that you can’t add a separate instance of it (which is useful for other actions like autophase)
CSHIFT operations with shifts less than the vector size currently do nothing. They should shift with an amount = (shift % vecSize) and adjust the reference by the full shift amount
Typing a value in the reference field (of Parameter panel) doesn’t trigger an update of the processing for indirect dimensions.
Users often want to be able to see what dataset is displayed in the windows. Make show title the default.
Increase precision (number of digits) in OrderPar output (toString) from 2 to 5.
Add chain and residue name and move frequencies to a column rather than in header
GRINS terminates iterations when the max intensity is less than the noise estimate times a noiseRatio. That ratio is currently hard-coded, but should be user definable (from script/GUI)
Default to csv in order parameter and spectral density output
Clicking process with no operations results in an error
Add a Tool for more control over peak picking. Include features that were present in NvJ including controlling mode (New, Replace, Append), region (Box or Window), local noise calculation, and filtered picking (based on another list)
Add menu items etc. to allow creating a molecule from a SMILES string
Automatically link peaks by labels when the peak list is generated with peak simulator. This will make it so users don’t have to explicitly call “Link By Labels” when generating peaks for use in peak slider. Only link peaks from peak lists that were generated with peak simulator
Changing phase values by typing into the text field (next to sliders in phase operation) doesn’t have any effect.
When searching for a nuclei in an axis label we should only return commonly used nuclei. So if you find an N, return 15N, not 14N.
Improve performance of GRINS (NUS processing)
Display shift probability (relative to prediction) and secondary structure prediction probability on RNA secondary structure viewer
Add zooming to secondary structure viewer so for large RNAs the user can zoom in on a region of the secondary structure
Connection lines between atoms can be drawn on secondary structure plot for each peak in a peak list, but this doesn’t work if the assignment labels have a residue name prefix. 32.H2 will display bug U32.H2 will not.
When reading in a new molecule (from file or Sequence Editor) the user should be warned if a molecule already exists. A common mistake is to read in a molecule twice. There should also be a way to clear existing molecules.
Bruker files have an FnTYPE parameters that is set to 2 for NUS files. Currently we assume its NUS if there is a file named nusfule, but it is possible for that file to exist, but the data to have been collected with uniform sampling. Using the FnType parameter should be more reliable
Simplify Resonance, SimpleResonance and AtomResonance to just AtomResonance
Simplify Project classes to ProjectBase, StructureProject and GUIProject
The scale for 3D viewer (camera position) should be initialized to an appropriate value so molecule mostly fills window.
Add ability to extract indirect plane to a .nv file and then read and use NUS processing on just that plane. This would be useful for optimizing NUS processing etc.
GRINS should be able to get NUS schedule from the matrix object (same way as NESTA now does. Useful for optimizing NUS processing.
The x axes of XYCanvasBarCharts often have fractional tic axis labels which can be inappropriate (for an axis representing residue numbers, for example). It should be possible to set an axis so the axis labels are always integers.
SF currently rounds to 1 significant digit which can result in incorrect SF in script. Also, when loading script the parameters that are set in script should be unlocked so they get written out to new scripts.
Adjusting planes to display a range of planes requires holding shift key down while adjusting spinners. This is not obvious to user. Should change so that left spinner moves the planes together, and the right spinner changes the second plane independently
Peak picking 1D spectra from the new Peak Pick tool should calculate a threshold based on estimate of noise level.
When Windows used for RunAbout are reloaded from .yaml file the RunAbout controls should be set up.
Update default options for NUS algorithms
If multiple datasets are displayed in a window they should all be picked from the new peak pick tool
Add Goto Peak, display Atom in table, fix starting from scanner table without loading table file first
nmrfxa and nmrfxs have add-opens (used by JavaFX) but they don’t need javafx to run.
Optimize GRINS options based on nuscon tool analysis
Add new RNA data (BMRB/PDB) and update training of the prediction models
Bruker weighted acquisition experiments use a vclist file. Loading this can prevent saving parameter file
Scanning for corrupted rows fails with an OutOfRangeException
A previous commit added a resFactory.cleanResonances to the NMRStarWriter class. It should be placed before writing the peak lists out. Otherwise the resonances in peak save frames and resonance save frames are inconsistent.
Loading a .yaml window file give this error and peak label types aren’t set properly:
2023-11-01 15:47:14,184 o.n.p.gui.spectra.PeakListAttributes lambda$setPublicPropertyValue$19
[WARN] Trying to set peakLabelType to Cluster but property isn't publicly exposed
Distance limit slider is missing from NOESY section of Peak Simulator
RNA secondary structure prediction is important to several NMRFx actions with RNA (structure calculation, shift prediction etc.). It would be useful to be able to predict the secondary structure inside NMRFx. This is a test feature and requires a model file that is not yet included in the release.
Initially, this will be an experimental feature dependent on a separate download of a trained tensor flow model
You should be able to undo most interactive peak actions (moving peak, adding peaks, changing multiplicities etc.)
If a user has an open dataset they should be able to open the FID that the dataset comes from (assuming it is present and can be located) and be able to reprocess the FID. This is facilitated by the fact that .par files (for NMRFx format datasets) now contain the location (in absolute and relative formats) of the FID they were derived from
Fix RNA peak generation with labeling scheme and allow selecting intensity of RNA peaks to be displayed on SS plot
When using the slider to assign NOE lists one should be able to specify that only frozen peaks are to be used in generating constraint lists
It should be possible to close the last stage on MacOS and keep the app open (as the screenmenubar still exists to give access to creating new spectrum windows and other GUI windows).
The ZZ exchange tool should also support A-B exchange (like folding/unfolding) besides the current protein-ligand binding
Some NvJ users new to NMRFx would much prefer the mouse scrolling behaviour of NvJ, where scroll would pan the view vertically, shift-scroll would pan it horizontally, control-scroll changes vertical scale, and option-scroll zooms. This behaviour is consistent with some graphic design programs. We should add a preference to switch between the old style and new style (which was chosen for consistency with SpinIt)
The ScanTable allows you to process to multiple datasets, and process and combine to one dataset. You should also be able to assemble already processed datatasets into a single (pseudo-2D or pseudo-3D) dataset
Can’t use EXTEND operations in simulated FID
The fillOval operation is not currently supported in PDFGraphicsContext. It can be used by Symbol.draw and its lack prevents exporting graphics with filled circles.
For example, Residue Order Value in the sequence canvas uses filled circles and can’t be exported.
Add support for fitting ZZ exchange data
The peak content display isn’t always consistent with current peak lists
If you add a dataset to a chart with scanner table open the table does not get updated to include the new dataset
When resizing a spectrum window the split pane adjusts so that the lower pane is visible, even when there is no content in it. The size should remain constant (whether it is empty or when occupied with something like the scanner table)
The peak simulator generates peaks with widths that look tiny on typical spectra. We should make them larger and related to the size of molecule so larger molecules get wider peaks
In the current release the initial stage is an appropriate size on some platforms, but too large on others. The size of screen should be detected and the stage size set based on that.
Users want to be able to add simple annotations, lines, arrows with text labels, text, rectangles etc. A GUI should be added that allows creating the annotations and controlling attributes (colors, line widths etc.)
Add button trigger reloading of scanner table. Currently the table can be reloaded automatically which sometimes removes dataset rows inappropriately. The user should have control over when it is reloaded.
NMRFx has a complex method of drawing spectra that exists to so that drawing complex and multiple spectra uses multiple threads. This prevents the program from hanging while drawing big datasets and speeds drawing multiple datasets. But the overhead of the current implementation causes flickering when drawing even simple spectra. An immediate mode exists which doesn’t flicker, but there is no way to activate it. We should add a preference to allow turning it on. Ultimately it may become the default mode.
Opening a favorite doesn’t work (nothing happens
The scanner table File menu has a “Load from Dataset” item. The code for this item should execute automatically when you open a scanner table and the active chart has an arrayed experiment in it. If the chart has multiple datasets in it then the table should load with a row for each dataset.
Currently you can’t see the reference info (sw, sf etc.) for all dimensions at once. You have to switch dimension and look at each one individually. There should be a single pane (in accordion) for reference info and it should show standard parameters for all the dimensions, and additional experiment specific info like solvent and temperature.
We should also add a parameter for acquisition mode (hypercomplex, echo-anti echo) so it can be set in the reference pane, rather than as a Dim1 operation
Current annotations inlcue lines, polylines and text. We should add more (squares, arrows, ellipses etc.)
Supporting native execution on Apple Silicon processors will lead to improved performance.
JDKs for Apple Silicon processors exist, but some of our dependencies use C code compiled for specific processors which may not include versions for Apple Silicon (macos-arm64).
Change the list of operations to an accoridion with one operation for each titled pane. Prepopulate accordion with commonly used operations (which may be disabled initially).
Currently, the phaser (on right side of spectrum) works in both processing mode and on processed datasets. Phasing in processing mode is now done in the Phase accordion pane, so the phaser should be changed to only work in dataset mode. Ultimately, it may be moved to another location.
The attributes controller doesn’t currently allow setting the line width for traces
Continue improvements to scanner tool including, row selection should load FID if no dataset in row, highlighting should work for arrayed datasets, and selecting chart should not replace scanner table if it is currently active
Bruker datasets can have various types of lists of values (vdlist, vocalist etc.). One type is a list of frequencies. We currently support reading an fq3list file, but should also support fq2list
A common technique is to collect an arrayed experiment and then calculate the difference of rows. There should be an nd script method (like nd.diffRows()) to calculate a difference spectrum
Three dimensional experiments where the second dimension has TD size = 1, so it isn’t incremented are not currently supported. These are used in making filtered NOE experiments.
The atom table can be filtered by residue number and atom name. It should also be possible to filter by residue name (like K.CA to show CA atoms of all lysines)
Small vectors may have insufficient numbers of regions identified as baseline causing a failure to identify any baseline regions and NaN values after baseline correction
A menu item should be added in console window to give ability to load and run user scripts in the GUI
You should be able to do Commd or Control-C in Peak Table, and use a Copy from an Edit menu to copy the table of peaks to the clipboard in a format that can be pasted into Excel
A table of peak lists should be added, similar to this one in NMRViewJ:
NMRViewJ allowed filtering the peaks displayed in the Peak Inspector/Tool by various criteria such as whether they are deleted or assigned. This should be added to NMRFx
Add a Go button (like NMRViewJ has) that sets the active chart to display the current peak (in inspector) or selected peak (in table).
The Peak Simulator can do many types of multidimensional experiments, but doesn’t do simple 1D proton and carbon experiments. This should be added.
The rnalinks test in PyStructureTest generates a structure of an RNA molecule and checks that the energy of the structure is below a threshold. The calculated structure can depend on random starting values so the energy can vary such that sometimes the test fails. We should raise the threshold energy so that failures are unlikely to happen, but major errors in the calculation are still caught.
Currently, the scanner tool (when working with multiple datasets, rather than arrayed experiments) controls the display of datasets on charts by adding and removing them from the chart. Instead, it should add all datasets to the chart and control what is displayed by turning visibility attributes on off. This will allow datasets to retain display attributes (especially color), as you add and remove them)
The slider tool should have the capability to of using a bipartite match to shift peaks from their predicted positions to the positions of peak maxima. Peaks should only be moved if they are in a specified tolerance and no ambiguity. Shifted peaks should be frozen so it is clear which peaks have been moved and which haven’t
Prediction of cross peaks in the TOCSY simulation misses many cross peaks even at the maximum number of transfers selected.
Improvements in scanner table loading. Show more error information if fid files can’t be loaded, allow grouping upper case names in table headers and avoid NPE in table that doesn’t yet have dataset entries
Use linkers to allow generating fragments of larger RNA molecules. Initialize angles for some standard RNA motifs (stem loops, simple bulges etc.)
Add basic NOE constraint calibration and export
Current peak fitting algorithms in NMRFx only properly work with Lorentzian line shapes. We should add support for non-lorentzian (perhaps a generalized lorentzian which can emulate gaussian shapes)
NMRViewJ supports selecting one or more rows in the peak table and clicking a delete button (and possibly delete key) and deleting the selected peaks. We should add this to the NMRFx peak table.
Non-phase sensitive files still index in groups of 2, like phase-sensitive files, which prevents use of NUS lists for them.
The input data (pressure values and ligand concentrations) should be sorted so that peak paths can be followed properly
Add more RNA secondary structure types for initializing 3D structures.
It used to be possible to delete all operations but now deleting all operations results in an IllegalArgumentException.
If the only operation applied to the FID is apodization, it is reversed between FID w/ Spectrum and Dataset views.
Autophase from doesn’t do anything in Spectrum mode
The xaxis is always H for FID w/ OPs when there is a FT operations (without it, the axis is time).
The GRINS method can fail for 2D datasets because the dataInfo.curDims variable hasn’t been set. This variable is unused so should be removed.
It’s possible to have a Varian experiment where a parameter is arrayed, but the arraydim parameter is set to one. In this case the array of data won’t have been collected (only the first row). We should check the arraydim parameter and if it is less than or equal to 1, assume the experiment isn’t actually arrayed
The HMBC chemical shift predictions (Peak Tool → Simulate tab → Experiment type → Basic ->HMBC with transfers set to 5) leave out many expected cross peaks. The lack of expected cross peaks makes using the Slider Tool difficult for completely assigning the spectra.
Changes in NMR-6326_delete_peaks_in_peak_table resulted in an endless loop when you edit a peak label in the Peak Inspector
We currently support predicting carbon and hydrogen chemical shifts using a hose-code like predictor. We should add support for N shifts.
Scroll wheel now changes level, not offset. Offset can be changed by scrolling in left border area of 1D spectra
Make the grid lines more fine as well as change the lines to dashed lines
Arrayed varian data (like T1 experiments) are not currently read in properly. Only the first row is read.
Generated python script starts by opening the file.
When the path contains special characters (such as french accents, for example "Qualité"), the file encoding is not correct, and the file can't be open.
In the Spectrum menu there is an item "Delete Spectrum". This deletes the currently selected chart (if there are more than one chart displayed) but the space is not reallocated, and can only be readjusted by using one of the options in the Spectrum menu item "Arrange". See screenshot of grid of 3 spectrums where the middle spectrum has been deleted.
Saving projects on windows results in a caught exception: `[ERROR] Creating XDG_CONFIG_HOME directory H:\.config failed java.nio.file.FileSystemException: H:\.config: Unable to determine if root directory exists`
The project still saves and can be opened despite this error. The computer this showed up on did not have an H: drive.
This link could possibly be related to the problem: https://git.eclipse.org/c/jgit/jgit.git/commit/?id=838b5a84b5093c335b95a644b8888006d9e95493
The autophase algorithm (flat mode) works by finding phases that symmetrize the baseline on each side of peaks. This criteria is independent of whether the spectrum is inverted or not. The code has a penalty function to choose the right-side up version, but apparently this doesn't always work right and the spectrum is upside down.
The pointToPPM method in VecBase is used by the ARTSY code, but is giving the wrong result, so the frequencies of the found lines are wrong.
excluded_jar_files_for_platform uses “mac”, not “macos” in test for files to exclude.
This means that essential .jar files are excluded from the Mac version of the application.
In particular, operations using the deeplearning4J library won’t work
Closing NMRFx with the X button after the regions table has been opened (regardless of whether it is currently shown or not) results in a NullPointerException.
The Operations VECREF() GEN() are used to generate some fake data. After the autoprocessing changes in NMR-6012, they no longer generate the fake data initially, only a flat line, and modifying the inputs does not change anything. But if you select FID w/Ops, you get a exception and then using/adding VECREF and GEN as operations will generate the fake data.
When a scan directory is loaded and the user attempts to process it, there is an illegal argument exception thrown.
If your process a dataset and then do anything that saves the .par file, without closing and reopening the dataset you get incorrect values in the “dim” line of the file.
Varian files with f1coefs = “1 0 0 1” are not processed correctly. NMRFx does not determine that the indirect dimension is complex.
If the axes have been changed from the default axes, ex. default axes are 1H and 13C and the 13C axis has been switched to 15N, then making a minor change in the operations list with auto process enabled or clicking the process button causes the chart to reload and the axes are reset to the default.
If the chart is in stacked display mode and the user attempts to process, a null pointer exception occurs.
Exception in thread "JavaFX Application Thread" java.lang.NullPointerException: Cannot invoke "org.nmrfx.datasets.DatasetStorageInterface.getFloat(int[])" because "this.dataFile" is null
The Next ProcNum button immediately generates a directory before the processing occurs. If the user presses it more than once then it will seem like in spinit many datasets are available. but switching to one of the empty dataset folders will give a file not found exception since it is empty.
Extension of indirect dimension FIDs is commonly done with linear prediction. An alternative is to use NUS processing where the schedule is automatically generated to correspond to the uniformly collected data with the non-collected data forming a uniform group of following entries. We should support this as it is potentially simpler in parameters and more robust
We can plot data derived from the scanner table, but should also be able to fit the data to various equations (like exponential decay and reaction kinetics). This would be useful for T1 and T2 relaxation as well as reaction monitoring.
When using some dataset types, like Bruker t1/t2 NMRFx can read in a list of values corresponding to an arrayed dimension. These can come, for example, from the Bruker vclist or vdlist.
These sometimes, especially for vclist, need to be converted into other values.
For example, vclist (which are integer counters) need to be converted to delay times for experiments like t1 or t2.
It would be useful to have a simple tool for doing math on columns with a formula like:
v = (v+A) * B + C,
where v is the original value which will be replaced with the newly calculated value. A,B and C are parameters to prompt the user for.
This tool could be prototype for more extensive math in places like the scanner table
Arrayed experiments should be displable as stack plots with controls for horizontal and vertical shift of each trace.
Beginning users have a hard time finding all the controls that affect spectrum appearance (colors, line widths, fonts etc.)
We should remove the spectrum attributes dialog and move the controls into a side panel where they will be more accessible. Controls should have better labeling so it is more clear what they do.
The attributes pane should be able to be hid so it uses less screen space when not in use.
Currently users have to click Process to go to a processed dataset they can work with.
For small datasets, 1D/2D, after automatically generating a processing script (or loading an existing one) the data should be processed and the GUI put into Auto Update mode so changes to operations (like anodization) immediately reprocess the data and update the chart with the fully processed spectrum, rather than the FID with operations applied.
NMRViewJ and dataChord both had additional cursor modes (beyond crosshair and selector) for operations like interactive peak picking and region addition. These capabilities should be added to NMRFx
Some pseudo-3D relaxation datasets have the relaxation delay on the 2nd dimension. After processing the 2nd dimension is not a frequency dim, and the dataset doesn’t display. We should allow these datasets to display properly.
We should also start work on allowing swapping of dimensions during processing so that the 2nd non-freq dimension data is swapped with the 3rd dimension data so the dataset ends up in a more conventional format
Structure calculation is done in torsion angle space which requires the structure to be generated as a long tree of bonds and rotatable angles. Generating large RNAs would be easier if the tree was branched so that residues that end up being close in space have fewer rotatable bonds between them. This could be accomplished by adding artificial linkers at the start and end of secondary structure elements.
Currently objects like datasets and peak lists are associated with a project, but molecule information is stored globally. It would be useful to associated molecules with projects. There are applications where having multiple projects open simultaneously with different molecules in each is important (working on multiple RNA fragments, for example). We can’t currently do this.
There is a table for displaying peaks, but no table for displaying integral regions. We should add this, including a way to save/load the table to/from a file
Sometimes data are collected where one or more rows of the acquired data are corrupted (for example, via power glitches during the acquisition). One way to salvage these datasets is to use NUS processing where the corrupted rows are marked as if they were not collected. The NUS processing then replaces the corrupted values with estimated values.
We should support this in NMRFx (we've had specific requests for this feature).
It is currently possible to add projections to the dataset using Right click on spectrum-> Extract Slice/Projection -> Project. At this point the border size does not adjust correctly to fully display the spectrum, the top is cut off, see screenshot 1.
The projections can also be added through Attributes -> Data Selector and then adjusting the border thought Attributes -> Style, but this requires both the refresh and the full button to be clicked before displaying.
The height of the projection spectrum can be controlled using the Attributes -> Slices -> Scale option which is not intuitive.
There is also no way to remove the projections once they are added. There should be a command to remove them that will restore the borders to the default.
Right now the projections are scaled at the same time using Spectrum Attributes -> Slices and adjusting the scale. This is not the best because it doesn't allow adjusting the scale separately and it does not allow the user to adjust the scale independently.
Necessary for spinit’s interoperability.
The row/plane controls used for stepping through array experiments could be improved.
Change Y to Row
Don't reset to row 1 when clicking in spectrum. Currently if you set display to full, and then click in spectrum it returns to just the first plane.
Display row/plane numbers starting at 1, not 0, as user facing tools should preferably use 1 indexing as that is more typical of NMR use (row 1, not row 0)
If the dataset axis is in pt mode (not ppm) display plane numbers not ppm
If the dataset has values (for example, from Bruker vclist or vdlist for the axis corresponding to the spinner, display the value
Currently datasets can be opened with "Open Dataset" and fid can be opened with "Open FID". They should be combined into a single "Open" command and should automatically detect which file the user is trying to open. Some file types may be figured out through the filename, others may need to open the file first. It would be good to set this up in a way that other types of files could be added easily in the future (ex. .mol, .pdb) but they do not need to be added in this issue.
When loading datasets in the peak path tool it would be useful if corresponding peak lists are also added.
Currently, during processing (especially in auto-process mode), the dataset is saved to disk and then reloaded. Reloading the file can conflict with ongoing display operations (in other threads). There is no real reason to reload the file, saving is done only to persist the data when the program quits. So we should change the code to periodically save the dataset to disk, but use the current in-memory version for display.
It would be useful to be able to mark ResonanceSource objects as deleted. This would be used in RING to interactively mark items in the ResidueChart as being deleted so users could ignore “bad data values” without going back to peak lists and reanalyzing.
Dragging and dropping a dataset with higher dimensions than the currently displayed dataset causes an exception. Example drag and drop 2D onto 1D, or drag and drop 3D onto 2D. (This happens both dragging and dropping files and from the loaded datasets menu)
The scanner tool is still experiencing the same problems as described in issue NMR-5486 for JCAMP and RS2D datasets. The previous issue was only tested with .nv datasets. This issue should be tested with atleast these datasets: JCAMP, RS2D, NMRView (.nv), and Bruker dataset
Forward backward linear prediction works by calculating coefficients in the forward and backward direction and then averaging those coefficients. The coefficients are checked by rooting a polynomial and reflecting roots that are inside the unit circle. Artifacts are sometimes seen and some spectra that disappear if one uses forward or backward linear prediction but are present when one uses the average coefficients. An alternative that eliminates the artifacts is to extend the data with the forward coefficients and with the backwards coefficients and then average the extended data rather than average the coefficients.
Opening a spinit FID in nmrfx 11.3 seems broken. The data opens, the generated script looks sane, but the processed result is plainly wrong.
We need to see when this started to fail. Possibility that this was broken when introducing the “dataset” library?
NMRF.bat is not inside the installer
java.io.IOException: Cannot run program "C:\Program Files\NMRFxAnalyst64\nmrfx.bat": CreateProcess error=2, The system cannot find the file specified
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
at rs2d.nmrfx.NmrFxController.startNmrFx(NmrFxController.java:139)
at rs2d.nmrfx.NmrFxController.connect(NmrFxController.java:50)
at rs2d.nmr.gui.panel.NMRStudyExplorer.lambda$initNmrfxTool$29(NMRStudyExplorer.java:601)
at rs2d.commons.gui.action.BasicAction.actionPerformed(BasicAction.java:114)
at rs2d.commons.gui.action.AsyncAction.lambda$actionPerformed$0(AsyncAction.java:34)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
at java.base/java.lang.ProcessImpl.create(Native Method)
at java.base/java.lang.ProcessImpl.
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:159)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
... 9 more
Since 11.3.0, there is an issue when trying to process a spinit data.
We can open the FID correctly after (NMR-5917).
However, when we click on “Process” to save the data, this error shows:
Notice the datatype on top.
The process itself is saved, we see a new proc number appear on spinit, and spinit display is refreshed correctly when nmrfx was opened from spinit.
The frequency domain field of a Vec should be set (true) when you do a FFT and unset (false) when you do an IFFT. When writing a Vec to a dataset, the Dataset.freqDomain field should get set.
Currently the status of freqDomain is not consistent. If the fourier transform is removed from the processing of a 1D, the dataset.freqDomain parameter is still set to true after processing and the axis changes to ppm see picture. Additionally, the value of the freqDomain parameter is correct in the par file (1 if ft in processing, 0 if removed).
When opening a dataset into NMRFx from SPINit and using the process button.
The resulting spectra is displayed on a really small ppm windows.
After DEEEEEEP investigation, NMRFx use the spinit ppm spectral width and…reconvert it into ppm…so devide it by the Base frequency.
As a esult the frequency is “Base frequency times” too small.
If an FID is processed while the contents panel is open, it causes javafx.concurrent.Service WARNING Uncaught throwable in javafx concurrent thread pool likely because a non javafx thread is trying to update the GUI. This does not occur with the Attributes or Phases panels
NMRFx can import sparky assignment files, but it sometimes fails. The code checks for “Assignment” in header, but fails if there are spaces before Assignments. It loads assignments for 2D lists, but fails for 2D lists, and fails if the file has a Data Height column.
Chemical shifts for methyl atoms aren’t set in the molecule when reading from a bmrb file
At the start of processing NMRFx needs to create the output dataset in its final size. At present this is done by the code that executes the processing script. Each operation that can change a vector length is used to update a dataset size for that dimension. This can fail. For example, if operations are used that don't properly calculate size (perhaps user added python commands). Also, some processing sequences may result in dataset needing to grow and then shrink. This results in a processing error.
The code should be changed so that the whole dataset, not just vector along a dimension, can dynamically resize during processing.
Some prototyping for this exists in code before merging into nanalysis.
ChemDraw is a widely used program for editing molecules. Chemists should be able to copy a molecule from ChemDraw to NMRFx. One possible way to do this (as the ChemDraw format is rather complex) is to look on clipboard for a mol file. It seems from this code that ChemDraw will put a mol file on clipboard:
NMRfx supports loading file as FID or as Dataset from its toplevel menu. Dataset isn’t implemented yet for JCamp. This prevents loading spectra files from the 60MHz NMReady.
Now that NMRfx uses slf4j + logback, we’d like to have a log console available.
A processing script should be generated automatically when FID is opened so novice users don't have to set up any processing script
In the same way you can use drag and drop to drag an NMR dataset onto a spectrum window, you should be able to drag and drop a molecule file (.mol, .sdf etc.)
The scanner tool needs improvements for working with pseudo-3D files. These are created when 2D files are processed in the Process and Combine mode. The display should switch to full mode in the 3rd dimension so that selected rows are displayed and peak picking should only happen on the 2D planes (as 2D peak lists)
Some issues with projects are creating problems for users. When loading a new project it would be useful if the program warned the user if project type data (datasets, peak lists, molecules etc.) are already present.
Closing a project (so the above warning, when added, doesn't happen) doesn't clear some molecule related items. In particular, even though the list of molecules is cleared, the active molecule is not nulled and the AtomBrowser (if open) still shows atoms.
Relaxation parameters for deuterium methyl groups need to be read to/from STAR files.
Also, when writing out deuterium relaxation there needs to be more characters in the atom fields to handle atom names like HD11.
The current region and multiplet toolbars could be removed and instead use contextual popovers.
Spinlab can store frequency related parameters such as spectral width either in hertz, or in ppm.
Currently, NMRfx assumes that this parameters are always stored in Hz, but that’s not necessarily true. It is being changed by NMR-5110.
When a parameter is stored in ppm, there is a reference (by uuid) to a frequency parameter used to do the conversion : it is not necessarily the base freq.
The structure calculation features are largely designed to work with proteins, RNA and DNA and rely on a built-in library of template files for each residue (amino acid or nucleotide). Users (especially pharmaceutical groups) want to work with unnatural amino acids. Creating a new template (.prf) file is very tricky. It should be possible to read in a molecular structure in some other format (.mol, .sdf, .mol2) and have NMRFx create a .prf file
The standard BMRB STAR file OrderPar saveframe doesn't save info on the number of values and parameters used in fitting. This information is necessary if one want's to calculate various statistics (reduced ChiSq and AIC) after loading in a saved project. So we should additional tags to the OrderPar loop.
Peak prediction exists for various types of 2D and 3D experiments, but not for simple 1D experiments.
This should be added.
The zoom in button uses a magnifying glass with a minus and the zoom out uses a magnifying glass with a plus. This is opposite to every zoom in/out button Ive seen. The icons should be switched. Note; These occur in at least 2 places
Saving a table of relaxation parameters (R1/R2 etc. does not include deuterium related parameters like RAP and RQ
The Multiplet Popover tool has buttons for:
Extract: move a multiplet component out of the existing multiplet to a new peak/multiplet
Transfer: move a multiplet component from one peak multiplet to another
Merge: merge two multiplets into a single multiplet.
These related actions may not be working properly currently.
A protocol to test them should be setup and they should be fixed as needed.
There is now (after moving Region tool functions to contextual popup) no way to delete an entire region.
There should be a Delete button in popover tool. When adding this we should also change the Split button to an iconic button so the tool is similar to the Multiplet tool
The Molecule button on lower toolbar allows you to display the active molecule (or load a new molecule if none present) on the spectrum. But there is no way to choose which molecule (when more than one are in memory) is the active molecule.
Molecules, text etc. are objects on the canvas. You should be able to activate object so handles that allow resizing are displayed and object can be moved on canvas.
Text (multiplet analysis can't be moved or width changed). Molecules can only be currently resized with a zoom-in/zoom-out menu.
Adding operations can be complex for beginning users. We should:
1) Change label on add op menu from "+" to "Add Operation"
2) Add a Delete button to delete operations (currently only done with Delete key)
3) Move common operations to a "Common Operation" submenu and have an "Advanced Operation" submenu
4) Add an APODIZE operation that allows lb,gm,sb so its easy to switch between those common functions
5) Add a SUPPRESS operation (easier name to recognize than TDSS) with simpler parameters than TDSS
6) Add A BASELINE operation that simplifies and combines AUTOREGIONS and BCWHIT
The Processor sidebar can't be displayed unless there is an FID present.
But the Processor can be used for generating simulated data and experimenting (generally for educational purposes) with different processing operations.
We should allow the Processor to be displayed without an FID.
The Strips Tool allows a user to set up a grid of windows (with one or more rows) and multiple columns.
The basic infrastructure works in NMRFx, but there are minimal or no features for sorting the data used to drive the strips. Here is a request from a user:
As far as I can tell, it’s not possible to use a 2D peak list (e.g., HSQC) to pull strips. No error messages, but basically I can’t select x and y dims, and/or nothing happens.
It would be nice if changing the control list didn’t automatically change the Dataset
It would be nice if the control list name remained displayed after selecting – there’s no way of telling what is the current control list without selecting it.
It would be helpful to be able to set the Start and N values to specific numbers, not just change with slider.
I can’t detect a pattern, but sometimes when viewing strips the horizontal crosshair only appears in one strip.
From my earlier query:
Is there a way to change the order of the strips?
Display Z-axis shift with each strip?
Some Bruker datasets can have more acqu*s files than the number of dimensions. For example, there are 2D datasets that have an acqu3s. At some point we added (based on user advice) a check for the aqseq parameter inside the pulseprogram file. That should be present for 3D (and higher) datasets so if it was not there we could ignore acqu3s files and determine that the dataset is 2D. But a user has provided an example of a 4D dataset without the aqseq parameter so we are incorrectly considering it a 2D file. It looks like the $DIM parameter in pulse program could be used instead to check how many dimensions the dataset has.
Recent versions of Microsoft Word and Powerpoint (at least on Window) should allow pasting SVG from the clipboard, but this is not working.
We removed AtomResonanceFactory from the Java code, but it is still being imported in molpeakgen.py
That causes code using molpeakgen to fail.
The import should be removed and some tests for molpeakgen functioning should be added.
The tool menu on the bottom of the analyst window some of the options are duplicated, see screenshot below.
On windows, for some users, right clicking on the plot and selecting attributes generates an exception. The stack trace is included below.
When the Regions button is clicked and some analysis (regions or peaks) already exists the user is prompted before clearing the analysis. If they confirm the clearing then the existing analysis is cleared.
This works if existing regions were created by originally clicking the Regions button, but not if they were created by holding the Alt key down and dragging the mouse over a region. In that case no Analyzer object will have been created and an NPE will happen.
When using the grid display layout, FID are always opened in the first chart. This is not the case with spectra, which is correctly displayed in the selected chart.
Selecting the SpectrumFitter tool throws an IllegalAccessException (see attached stack trace) and only the one text entry area is displayed (see screenshot 1). If running from intellij, this can be avoided by adding the VM argument `--add-exports javafx.base/com.sun.javafx.event=ALL-UNNAMED` where the SpectrumFitter tool will not cause the exception and be fully displayed (see screenshot 2) .
Loading a 2D dataset after loading and processing a 1D FID, the undo button will cause an ArrayIndexOutOfBoundsException on the 4th click. See attached txt file with exception details.
When loading an FID using the command File -> Open FID, in the File Browser you can select an FID file with a prefix such as .dx. If a dataset file is selected instead (ex. with an extension .nv) an error message is displayed to the user and nothing is displayed in the chart but the logs indicate a dataset was loaded. The dataset can be seen from the dataset display
When loading a dataset using the command File -> Open Dataset in the File Browser, you can select a dataset file with an extension such as .nv. If an FID file is selected instead (ex. with and extension .dx) nothing is displayed to the user, nothing is displayed in the chart but the logs show that they parsed some data.
The Dataset class was updated with new methods for getting dataset size based on whether one wants the number of complex values or number of real+imaginary values. This script wasn't updated.
The Positive and Negative check boxes in the Dataset Table don't do anything
When there is a ProcessingException, the size of the processing tab changes size weirdly (see screenshot) For an exception, the text of the processing statusBar is updated to display the exception message, but not all is displayed since exceptions are quite long.
In addition to fixing the sizing, it might be useful to have a hover popup with the complete text, since the exception is cut off?
2D and higher experiments have an adjustable slider when the FID is loaded. The slider updates a textbox with the text format `{current value of slider} / {maximum value of slider}` but the initial value of that textbox for the 2nd dimension always seems to be different initially, then once the slider is adjusted at least once then, it updates to the correct value. If the FID has 3 dimensions, the slider for the 3rd dimension matches the correct value right from the start but the 2nd dimension does not. See screenshot
When exporting graphics, exporting svg or pdf will export the displayed chart, but exporting by png will display a blank image only. See three files exporting the same dataset. Note: Exporting the files saves them without a file extension, unless the user specifically types the file extension, which makes them harder/more annoying to open later in windows.
If the processor is working on processing an item while the ScannerTable is open, clicking on one of the rows in the ScannerTable during processing will result in an error message, with a red circle in the processing status and the row FID being displayed.. The processing progress bar continues to update, once it completes the red circle turns green but no dataset is loaded. A processing python script is created with the contents of the processing file but with the wrong name/location.
If a user loads a dataset and selects Tools -> Protein Tools -> Show RunAbout when there is no molecule loaded, an exception occurs when the tool is loaded. Any attempt to close the tool, change the window shape, load the processing window will result in incorrect formatting. The screenshot shows the gui after this exception occurred and then the window was maximized and a FID was loaded. The exception is also attached. The only way to fix this is to restart the software, or to go to Spectra -> New Window and then close the old window.
When the software is first started, or after a dataset is loaded/processed, the bottom menu tools bar is available and Tools -> Analysis (1D) -> Show Multiplet Tool will add the multiplet tool bar to the bottom of the screen and display 3 pop up messages, If you continue to select to Show Multiplet Tool, additional multiplet tool bars will be added to the bottom of the screen.
When multiplets are first applied they line up nicely see left picture, but if the multiplet tool is opened and the multiplets are selected using the selector tool, the spectrum, multiplet and xaxis shift so they are no longer aligned correctly, see centre and right screenshot.
Regions can be deleted using the regions tool. If regions are present on the chart but none are highlighted in the plot, nothing will happen. Selecting a region and then deleting will delete that region, any subsequent clicks of the delete button without a region being selected will start deleting regions one by one from the left of the last deleted region until it gets to the last region which will result in a NPE and the region will remain drawn on the chart. Trying to select that region will cause it to disappear.
From the Regions button (1) , the regions are printed with normalized integrals. For the Find Regions menu item (2) in the region tool, if no prior regions have been made the integrals are not normalized, otherwise the integrals are normalized.
The path tool has two sets of arrow buttons, the right set are bigger than the left ones, it will look better if they are the same size.
If processing is halted before completion, and then a new FID file is loaded and processed, it can cause a processing exception. This doesn't happen all the time, so it might be related to the timing of when the processing is cancelled.
The RefManager in setupItems and getSkip parses strings from the script.
If the string has a valid integer it uses that value, if not it uses a default value.
There used to be an empty catch (NumberFormatException), but now if the value isn't a valid integer a warning message will be displayed.
We should properly check for acceptable non-integer values (typically empty string) and only show warning if there is truly an error.
Clicking integrate on a 1D spectrum should show integral regions with normalized values. With (at least one) JCAMP spectrum the integrals are flat and norm values 0.0
The type of cursors (typically selection arrow or crosshair) is controlled by the Cursor menu at left side of spectrum status bar. Sometimes the displayed cursor is inconsistent with the menu selection.
The dimension choice box for 1D FIDs is not consistent. If a 1D FID is loaded after the software starts, the dimension choice box is empty.
If another dimension FID is loaded, and then a 1D FID is loaded, the dimension choice box is not updated and will contain the options for the previous dimensions and will result in an exception if a user selects a dimension greater than D1.
Selecting multiple rows in the scanner table should show those rows in the spectrum chart. If you click in the chart the displayed row gets reset to the first row.
The recent addition of a check to see if files open from the "Open Dataset" menu are Datasets, not FIDs, results in the inability to open many .nv files. NMRViewJ/NMRFx (.nv) files created in other software (exported from nmrPipe, for example) do not have freqDomain flags set. Because of this the isFID check returns true, even though they are not FIDs. We need change the isFID check to properly handle these files.
When saving a project .nv and .ucsf datasets should be saved as hard links to the original file.
.nvlnk files are currently used for other dataset formats.
But, in present code .nv and .ucsf files are not properly detected (checking Path endsWith not String endsWith) so files are always saved with .nvnk files
Before the peak slider tool is opened, peaks can be selected, After it has been opened and closed, peaks can not be selected. Only the peak sliding functionality works.
Loading a molecule from a mol file and adding it to the canvas in NMRFX vs ChemDraw, the molecules appear to be flipped on the horizontal axis. See image
On windows, setting a favourite dataset results in an exception. The filepath printed in the traceback has interpreted the directory vcs as `x0bcs` where x0b is the vertical tab character.
```Exception in thread "JavaFX Application Thread" Traceback (most recent call last):
File "
File "_pyclasspath_/nwyaml.py", line 11, in dumpYamlWin
IOError: (13, 'Permission denied', 'C:\\Users\\Jannalie.Taylor\\Documents\x0bcs\\llll\testtest\\windows
MyDataset_fav.yaml')```
Trying to favorite the dataset when there is no project loaded causes an exception. Attached is a stack trace.
If a spinlab or JCAMP processed dataset is open when the project is saved, they will not be saved to the datasets file. Opening the project will print out a message saying the dataset is not there (its not a log message) and nothing will be displayed.
In addition to saving the dataset, add log messages so the user can check the logs.
The installer generator ( generate-installer.py ) excludes various jar files from the mediaSets for each platform. This works for windows and linux, but excludes some files from the macOS mediaSet that should be included (openblas, javacpp ..). This is because the platform name passed in to excluded_jar_files_for_platform is "macos", but the code looks for "mac"
Opening and closing projects with more than one window and then hitting saving, will increase the number of windows opened on the next time the project is opened. Looking at the steps to reproduce the bug, if steps 3 and 4 are repeated multiple times before step 5, then the number of extra windows opened in step 7 will be increased.
Drag and dropping datasets onto the canvas does not work for Spinlab and JCAMP datasets and does not work for Spinlab FID.
The drag and drop for JCAMP tries to load the dataset but gets a "Use Open Dataset to open non-fid file" error message.
The drag and drop for Spinlab datasets is not enabled, there is a greyed no symbol displayed indicating the file cannot be dragged and dropped.
NMRfx uses install4j to build installers. The scripts used for that are not in the “nmrfx” repository for now. They probably shouldn’t, because we can keep the installer as a reason to buy nmrfx, but they should be in “nmrfx-private” repository.
The second problem is that install4j is a paying software. They have an option to buy a headless runner for CI/CD pipelines, that could be enough for us.
We want to migrate to java 17 (Azul zulu with javafx modules included).
This includes pom modifications, changes in github build, but also updating the installer.
Spinit JCamp export has been redone in the past few months, and have been tested successfully both on Mnova and Topspin.
Initial tests showed that some of them didn’t create the correct default processing script and ended with a wrong FT or inversed data. We should check them and fix JCAMPData accordingly.
The test files are available on sharepoint/teams:
Remove the module files, remove the IDE specific launchers (they shouldn’t be needed), see if other changes to startup scripts may be needed.
Remove the dependency to ejml-fat and use a normal, maintained, dependency instead.
IMPROVE GUI Allow gscript datsets to use dataset object or dataset name
NEW CMD Add extract method to dscript to extract a vector to a vector dataset
BUG CMD Setting up regions in SimData was overlapping regions that should not be The original method assumed regions were created in ppm order. Now search through existing regions to check for overlap
BUG GUI Fix setting up mapToDataset when P1 methods present. Phasing was failing
IMPROVE GUI Make Script pane non-editable. Will add a control to turn on/off
BUG CMD Fix setAcqOrder in JCAMP. Reading JCAMP files was failing
SKKKKKKKKKKkiP IMPROVE STR 1631222264 Update RNA attribute prediction data
IMPROVE CMD Change BucketedMatrix to use double[][] for portability with Apache Commons Math * SMILE
BUG GUI Label color of peaks wasn't being set properly
SKKKKKKKKKKkiP IMPROVE STR 1628172551 Allow generating hose codes on polymer
BUG GUI Fix reading vector for 1D
BUG GUI Fix error with vector dims selector on 1D
SKKKKKKKKKKkiP NEW STR 1627911261 Add readMol2 to molio.py
IMPROVE GUI use getDlabel (instead of getLabel) in Dataset to get label for chart
IMPROVE CMD Change getStdLabel to return greek delta followed by supersripted nuclei
IMPROVE CMD Add toLatextString to Nuclei. REturns a superscripted latex format
IMPROVE GUI Change from spinner to choicebox for real/imaginary selector
IMPROVE GUI Move vecSpinner from status bar to be with row controllers
IMPROVE GUI More flexible sliding of FID display
IMPROVE GUI Add export TRACT data
SKKKKKKKKKKkiP IMPROVE STR 1623956842 Newly trained 2ndStr model
BUG GUI Fix loading indirect vectors that are not complex
SKKKKKKKKKKkiP NEW STR 1623776461 Add prediction resfactors and model for 2nd structure prediction SKKKKKKKKKKkiP NEW STR 1623714902 Add Protein2ndStructurePredictor
BUG CMD Fix writing ucsf header when saving memory files
IMPROVE CMD Add negateOdd to rft. And detect whether to use rft in autogen
NEW CMD Only adjust dwell if size different and not frequency domain
NEW CMD Add TPPI acq type to RS2DData
BUG GUI Fix problem with save/load stages on Windows. File path corrupted
SKKKKKKKKKKkiP BUG STR 1623034792 Allow names like stage_0 instead of 0_stage
NEW CORE Update to Java and JavaFX versions 15
NEW CORE Read T1/T2/NOE and order parameters from STAR files
NEW GUI Add new sequence display GUI with atom shift deltas and order/disorder calculation display
NEW STRUCTURE Add random coil chemical shift estimation based on concepts from Potenci
IMPROVE GUI Updates of code for peak path (titration and pressure) analysis
NEW GUI Add support for cd,pwd,ls commands to console
NEW GUI Add new simple Peak Assign Tool to toolbar at bottom of spectrum for entering peak assignments
NEW GUI Draw triangles at side of chart to show pivot position.
IMPROVE GUI Updates to interactive phasing controls.
IMPROVE GUI Set dataset name for processed dataset from an existing .nv/ucsf files or script in the FID directory
NEW GUI Automatically load an existing process.py script in FID directory
NEW STRUCTURE Add estimation of steric alignment tensor based on concepts similar to PALES
NEW PROCESS Add initial support for data files from RS2D consoles
IMPROVE STRUCTURE Update RNA shift training to use LASSO regression.
NEW GUI Add Projection display to show 1D projections on edges of 2D spectra.
NEW CORE Add tools for pulling data from remote server. Currently prototyped with nmrbox.
NEW GUI Add dataset browser to find NMR data (datasets and FIDs) in a directory and load them with button click.
NEW GUI Allow reading chemical shifts from a star file
NEW PROCESS First version of psglib for a library of processing scripts
NEW STRUCTURE Initial command line option for measuring PDB files.
NEW STRUCTURE Add doing ligand (small molecule) predictions
NEW STRUCTURE Allow output of rna attribute prediction information
NEW STRUCTURE Add -n option to generate a nef file from loaded restraints
NEW GUI Allow sliding in pseudo-3D datasets (like T1/T2). Uses first plane Should change to autodetect displayed plane
NEW GUI Add RING NMR Dynamics GUI to NMRFx
IMPROVE GUI Region measurement (from Scanner Tool) only worked with pseudo-2D. Now works with multiple 1D files.
IMPROVE GUI Move setting of scan directories (Scanner Tool) to a preferenes pane (from menu items)
IMPROVE GUI Export slices (if currently displayed) to svg/pdf files
IMPROVE GUI Move clustering menu items to separate Cluster menu
IMPROVE GUI Draw lines between linked peaks using peak list color (they were always black)
IMPROVE GUI Allow turning off/on peak listening in PolyChart. Changing lots of peaks could slow display way down.
IMPROVE CMD Add code to remove diagonal peaks
IMPROVE GUI Show exception dialog if fid file not found rather than fail silently
IMPROVE GUI Clear datasets and remove datasets/peaks from chart when closing project
NEW GUI Add ability to extract vectors from chart and display them in a new chart that pops up.
IMPROVE GUI Add fit all and add all to peak path tool
NEW CMD Path Tool: Write all paths, not just ones with pars, to star file. This way we track parameters like confirmed on all paths
NEW CMD Fix saving Peak Path (titrations) to star file
NEW GUI Get display of clusters working for RunAboutX
NEW GUI Add Filter menu item
NEW GUI Add display of amino acids consistent with cluster shifts in RunAboutX.
NEW GUI Add panel for shifts for atom types for SpinSys in RunAboutX
IMPROVE GUI Speed 1D drawing with lots of multiplet labels
BUG GUI Fix control-scroll changing of 1D vertical scale
NEW CMD Fix saving multiplets to STAR file
IMPROVE CMD Add more multiplet conversions (to m, to s etc.)
NEW CMD Read/write norm from/to .par file
NEW CMD Write peakList scale to STAR file
NEW CMD More work on 1D analysis including multiplet info to STAR file and displaying journal format
IMPROVE GUI Work on analyzer functions split/extract/merge. Add support for mol menu
NEW CMD Work on 1D analyzer functions split/extract/merge
NEW GUI Add picking and moving molecule on canvas
IMPROVE GUI Lighten Ligand canvas background color
IMPROVE GUI Add Molecule and Region tools to SpectrumStatusBar tools
IMPROVE GUI Move Multiplet Analysis to tool in SpectrumStatusBar
IMPROVE GUI Move more tools to SpectrumStatusBar. Add method to add tools
IMPROVE GUI Get PDF Export working
BUG GUI Fix exporting large datasets to svg. Peaks were appearing in wrong postion in SVG output.
BUG GUI If one closed all spectral chart windows you couldn't open a project. Now a new stage with chart will be created on demand.
BUG GUI The nw.grid command would create a grid of windows, but they wouldn't appear immediately.
NEW GUI Add slider to control lvl,clm,nlvl,offset in the Spectrum attributes dataset table. Use a choice box to choose which parameter will be controlled. This applies to the selected dataset rows or all rows if none are selected.
NEW GUI Add aspect controls (slider in SpecAttrs) in view section of Spectrum Attributes. The axis shift range is scaled by the axis nucleis gyromagnetic ratio.
IMPROVE CMD Add reading sparky assignment (in Peak Tool File>Read Peaks menu). Type of file is automatically determined based on extension and content.
IMPROVE GUI Add "vp" pop view binding. Pops current chart into new toplevel chart.
BUG GUI Atom labels in peak navigator should use dim on x and y axis. Was using first/second dim of peak
IMPROVE GUI Make Peak Navigator width scaling uses the mean width of peak list, not the individual peaks width. Allows consistent view across all peaks of list.
BUG GUI Fix Peak Navigator move to peak code. Wasn't accounting for whether planes were ppm or pt
NEW GUI Add initial version of nw.bindKeys for user defined key bindings.
NEW GUI Add AutoCouple menu item. Collapses adjacent (coupled) peaks into single peak.
NEW GUI Add "Add Mirror Peaks" menu item. Generates peaks across diagonal from existing ones. Two D spectra only.
BUG GUI Fix offset of 1d integral display. Wasn't accounting for borders
BUG GUI Display of apodization on FID wasn't using chart borders correctly so was ooffset from correct position.
IMPROVE GUI Set title on window when loading from .yaml files
NEW CMD Add calculation of some measures used in NusCon for comparison of NUS processed datasets.
NEW GUI Add err field for intensity and volume in Peak Inspector
NEW CMD Add err value to peak measures
IMPROVE CMD Read/write noise value in dataset .par file
IMPROVE GUI Allow setting dataset RMS from Analyzer window
BUG CMD NvJ par file stores ref pt starting at 1, not 0 Change for compatability with NMRViewJ
BUG GUI Auto level wasn't being called when opening a processed dataset
BUG CMD Multiplet analysis: Removing peaks until multiplet is a singlet didn't update peak shift and intensity
NEW GUI Add Peak Table to show all the peaks in a list
IMPROVE GUI Remove NvJ button. Not very necessary now and didn't always work.
BUG CMD Read condition from .xpk file. Reading failed if .xpk file had a condition line
IMPROVE CMD Auto add a TRIM command for direct dimension of Jeol file when generating script.
NEW GUI Add some methods to gscript to make it easier to script GUI from Python.
IMPROVE GUI Autophase and manual phase interacted poorly in GUI. Now the PHASE op is added after an AUTOPHASE op. The phase sliders start with the auto generated phases and the phases added to the PHASE op are the amounts that will be added to the autophase amounts.
IMPROVE GUI Remove yAxis drawing on 1D spectra
IMPROVE GUI Change drawing of 1D titles to stack in top left. That way they avoid overlapping spectrum
BUG GUI Add ColorBy column to peak tab and fix ColorBy so it works. Color peaks by plane offset, status, assigned etc.
BUG GUI Fix color of integral value and height of region fill
BUG GUI Fix displaying peaks by nplanes. Now, as in NvJ, peaks that are off the displayed plane are drawn in the "off color"
IMPROVE GUI Add regions checkbox and cleanup integral controls
IMPROVE GUI You can now pan spectrum left/right up/down with mouse dragging in left/bottom border area (on top of axis).
IMPROVE GUI Allow scrolling with mouse when shift+alt keys down.
IMPROVE GUI Add titles on spectra (controlled by checkbox in Spectrum Attributes)
BUG GUI Fix offsets so using offset column menu (ramp etc.) in spec attributes work
BUG CMD Fix problem with processing to ucsf file format (header needed updating)
IMPROVE GUI Fix positioning of 1D integrals. Now they are autoscaled to always fit on display within a region specified with a range slider in spectrum attributes.
IMPROVE GUI Improve bindings to allow user to use mouse in selector cursor mode to zoom and to add regions.
IMPROVE GUI Fix drawing sum of multiplet / peaks
BUG GUI Fix picking mulitplets
NEW CMD Add something like Objective Deconvolution to automatically add peaks to optimially deconvolve a region.
IMPROVE CMD Various improvements to Multiplet analysis including better fitting and gui improvements
BUG CMD Fix reading Bruker vdlist with m, u and s suffixes on delays
IMPROVE GUI Add .jdf file type as option when choosing FID files (Jeol file)
IMPROVE CMD Add new types of low-level storage for datasets. Use in-memory mode for processing if data small enough
IMPROVE CMD Improvements to GRINS. Added as an OP.
IMPROVE CMD Simplify dataset/fid size code in pyproc and Processor.
NEW CMD Add Merge op (merge alternate real / imaginary pairs) to Complex values
IMPROVE GUI Stop using .tmp file when processing (now that we don't use mapped file) Mapped file could be hard to delete (especially on Windows). Now using cache file for processing
IMPROVE GUI Update TRACTGUI to work with changes in TRACTSimFit
IMPROVE GUI MOVE cursor on Linux is closed hand, so use HAND cursor instead Should eventually add custom cursor (pointer cursor)
NEW GUI New tools for titration and pressure analysis. Based on new PeakPath tool
NEW CMD Allow fitting pseudo-3D peak lists. Shape and position are common for a peak accross all planes. Intensity can be independetly fit for each plane, or fit to an exponential decay model using delays associated with the dataset.
NEW GUI Reorganize Fit menu and add "Fit Planes" and "Fit Planes(Exp)"
BUG CMD Processing would fail on Bruker datasets if DSP charge-up values were zero.
IMPROVE GUI Add menu triangle and tooltip to dataset table dim column so it is more clear that values for different dimensions can be seen and changed.
IMPROVE GUI Change space binding so peak inspector moves on top of spectrum
NEW GUI Add protein backbone experiments (hsqc, hnco, hnca etc.) to simulate peaks tab
NEW CMD Add support of a fixdsp method to processing scripts. Allows turning on/off fixing of DSP charge-up (Bruker fileds) when reading datasets. This was only working in GUI.
NEW ANALYST Add ability to get RNA secondary structure from a loaded 3D structure.
IMPROVE GUI Add check box to peak list attributes table to turn on/off display of link lines.
NEW ANALYST Show molecule viewer after reading molecule in
NEW ANALYST Add command/menu to read simple rdc text file.
NEW ANALYST Color constraint lines in SSViewer by peak intensity classes
NEW GUI Add new spectrum menu items for clustering along rows/columns and fitting rows, columns and planes.
NEW CMD Allow constrained (column, row) peak fitting. All peaks in linked column(row) get same shfit and width in that dimension.
NEW ANALYST Reset molecule transform so initial rotation is consistent with pymol
NEW GUI Add support for lineshape pick/fit from menu
IMPROVE CMD Change to use layers of peaks when fitting. Fit includes center peak and surrounding layer of peaks. Next layer is not fit , but intensities subtracted from region.
NEW CMD Add new LorentzGausNDWithCatalog Line shape fitting using lineshape catalog
NEW CMD Add refinePickWithLSCat (start of something like INFOS)
NEW CMD Add linksDrawn (and get/set). Used in drawing link lines (lines between peaks that are linked)
NEW ANALYST Add original, molecular SVD, and RDC rotated axes in molecule viewer.
NEW CMD Add pick (peakpick) to dataset scripting (nd command)
NEW GUI Add GUI support for generating line shape catalog during processing
NEW CMD Add get method to return a dataset based on name
NEW CMD Add support for generating lineshape catalog. Generated during processing based on apodization etc. Read in when loading a dataset.
NEW ANALYST Add SVG plot export to RDC interface.
NEW ANALYST Add interface for RDC analysis.
NEW GUI Initial add of new path tool for following peaks in titration (etc.) experiments
NEW GUI Add tool for doing TRACT analysis
IMPROVE GUI Add button for closing dataset to datasets table window.
NEW GUI Get Styles tab working with settings for grid lines, colors and fonts
NEW CMD Initial creation of pscript to provide python interface (read/write etc.) to peaklists (npk command)
IMPROVE GUI Switch to using ppm for planes. Add nplanes tol to peak display
IMPROVE GUI Fix freezing of drawing with too many charts.
BUG CMD Close dataset when there is processing error. Fixes bug where processing stops working, even loading new file and setting correct processing parameters. Failure would happen, for example, with setting invalid (non-power-2) zf size.
BUG CMD Change spaces in dataset names to "_". Prevents various issues. Was a problem writing STAR files. We could quote it in STAR file, but seems simpler to work without spaces in dataset names.
BUG CMD DatasetPhaser wasn't saving correct phase values to dataset. Wasn't adding new phase correction to existing phase values
NEW GUI Add TRIM processing op to gui list
NEW CMD Initial attempt at getting Jeol Data reading working. Basic functionality, but needs work
NEW CMD Add TRIM op (to trim off edges of spectra)
NEW ANALYST Allow choice of dist or ring-current prediction for RNA in gui
NEW ANALYST Initial version of ligand scanner in NMRFx
NEW GUI Add menu item to save peak list to sparky format
NEW GUI Add button to save parameter file to Datasets table
NEW CMD Add Equation Fitter (with support for janino compilation) and use from Tract code
NEW CMD Add MatrixAnalyzer (used for PCA comparison of bucketed dataset)
NEW CMD Initial tools for new version of ligand scanner (analyzing effects of ligands on HSQC (etc.) spectra.
NEW CMD Add writePeaksToSparky
IMPROVE CMD When doing an alignmnent of 1D spectra calculate the alignment position of max intensity with interpolated polynomial. Alignment still done with integer shift, but report of fractional shift useful for analysis.
IMPROVE GUI When measuring peak intensities check if peak list has an associated dataset. If not, try to add one.
IMPROVE CMD When measuring 2D peak intensity/volume try to find a match of peak list and dataset dims if they don't have exact match.
BUG GUI Showing spectrum pop-up menu was deselecting any selected peaks.
IMPROVE GUI Auto add peak lists to spectra in RunAboutX
NEW GUI Add menu item (in spectrum pop-up menu, Reference section) for shifting peak lists to align with dataset
IMPROVE GUI Reduce the range of sliders for phasing (in both the Phaser tool and in the operations display) so that its easier to make fine adjustments
IMPROVE GUI Changing phase (ph0,ph1) values in the PHASE operation dialog now updates the sliders in the Phaser tool, and changing ph1 uses the pivot position set on active spectrum.
IMPROVE GUI Update Java to 8.0.202
NEW GUI Right clicking on the Color column of the Dataset table (in Spectrum Attributes window) now gives a menu item that allows you to set a color scheme that applies to multiple datasets (useful for titrations etc.)
IMPROVE CMD Allow using units (Hz, PPM, pts, Fraction) in CSHIFT
IMPROVE CMD Add adjref argument to CSHIFT and SHIFT. When set to true the reference value for vector will be changed to compensate for the shift.
NEW GUI Add new referencing methods and menu items, Set Reference and Shift Reference, to the spectrum pop-up menu.
NEW GUI Add a server mode that creates a socket-listener that can respond to commands from another program. Used for the CoMD dynamics software.
BUG GUI Fix showing new chart targets for adding new spectra to window. The bottom and top targets were missing since switch to canvas mode) mode)
IMPROVE GUI Add delete icons to data columns in the Scanner Table
NEW GUI Scanner table now has menu items for measuing regions in the pseudo-2D spectra. Similar to what in dCSM
IMPROVE CMD Set non-processed (arrayed dims to be not complex)
NEW CMD Allow specifying whether to generate script for arrayed dataset. It can be tricky to figure out if indirect dims are actually frequency dims, so this allows specifying that last dim is not a frequency dim and shouldn't have processing commands added in genScript
BUG GUI When closing chart remove its crosshair lines from plotContent Also, add moveTo that takes single axis data
BUG CMD Sweep was getting set incorrectly when reading vector from dataset.
IMPROVE GUI Import Python os module into console interpreter
IMPROVE GUI Scale window ppm (when making proportional windows) by nuclei g ratio
NEW GUI Can now disable crosshairs by clicking on icons in the crosshair ppm text windows in status bar.
IMPROVE GUI When minimizing borders of spectrum charts, now adjust chart sizes to be proportional to ppm range of the window (with some scaling by what nucleus is in use)
BUG GUI Align spectra wasn't picking correct peak regions.
BUG GUI When window is very small just draw a single tick/label at center
BUG GUI Minimize axes and grid/horiz/vert layout wasn't working properly
IMPROVE GUI Use getPercentile in auto scaling so that lvl doesn't get set too low.
IMPROVE GUI Reference gui updates. Fix C- to - F temperature conversion. Can now prompt for reference at crosshair position. H2O PPM mode inserts water ppm in script (rather than word H2O) C to F conversion was missing 0.15 (273.0 instead of 273.15)
BUG GUI Fix hitting (selecting) 1D peak label boxes
NEW CMD Add new SpinSystem code
BUG CMD Fix adjusting sw when vector not size of dataset
NEW GUI Add pop-up menus for peak, integrals and regions for 1D analysis.
IMPROVE CMD Add norm field to dataset to store dataset normalization value. Normalization used to translate integrals into number of atoms, for example
BUG GUI Fix drawing of integrals and add display of integral text value.
BUG GUI Get interactive adjustment of 1D regions working.
BUG GUI Get clipping for GUI display of peaklists working properly
NEW CMD When processing is complete on all dimensions set nFreqDim field to the number of processed dimensions. This is useful for pseudo 2D files
BUG GUI Drawing 1D vectors of pseudo-2D failed when display region smaller than 2nd dim
BUG GUI Scrolling in y-axis was backwards (after update to new axes code)
IMPROVE GUI Change Baseline drawing to be flat line at 0.0
IMPROVE GUI Add ability to populate the scanner table from an arrayed dataset
NEW GUI Add measure sdev and s/n
BUG GUI Fix spectrum axis syncing
IMPROVE GUI Increase preferred wdith of spectrum attributes window so its all visible
IMPROVE GUI Stop using popOverMode for spectrum attributes. A little flaky and hard to resize window
IMPROVE GUI Add separate Open FIDs and Open Datasets and Recent FIDS and Datasets menu
BUG CMD Fixes for setting coupling values, including fixes to removing extra peaks
IMPROVE GUI Add TextField to Integer Range displays in operation details
IMPROVE GUI New contouring algorithm using the marching squares algorithm. This generates continouse curves rather than a separate line segments.
NEW GUI SVG export now exports all the spectra charts in canvas. Used to just export the active one
NEW GUI Create new plotting that does spectra on a single canvas