difx:difx-todolist-2019
To-Do and Wish List
To be prioritised
Standing Items
- New python scripts should consider python3 conformance
- Documentathon with emphasis on inner workings (all)
- Update the utils page to show what package supplies each tool (all)
- Reconsider the approach…
- Update the pdf documentation for currently undocumented tools, and keep updated (Adam, WFB)
High Priority
- Determine why complex interlaced VDIF data is not fringing in DiFX-2.6.1 (CP, AD, JW)
- Change outputbands logic to choose appropriate spectral regions in overlap cases (JW)
- DiFX pcals with multiple datastreams (fix in difx2mark4). In progress. (HR, John)
- Mark6 group handing plan (WFB, JJ, MW, JW, HR)
- sanitycheckdifx: what files/issues need to be checked? Implement in vex2difx (CR)
- polyco typos
- binconfig typos
- addPhaseCentre typo in RA, or Dec, or whatever
- datasim: parallelize, document (Zheng)
- mark6meta library: fix issue of reporting “module complete” when using incomplete Mark6 modules (HR)
- DiFX library dependencies clean-up & move of functionality. Partially, perhaps fully, complete (WB,JW,MW)
- Regression testing scripts (send new datasets to Matteo)
- Plan: Matteo investigate Jenkins
- A more sophisticated pipeline than just diffDiFX?
- Target size: 10G
- Look for VDIF, Mark5B, LBA, zoom …
- vex2 support (relies on vex2 finalization and some support from field systems)
- document almost complete
- vex parser is almost complete
- Support for $BITSTREAMS largely complete in vex2difx
- Plan: finalise, test
- Mark6 playback speed (locally high, but via network speeds are slower)
Medium Priority
- Update vdifio to make the utilities compatible with Legacy VDIF (AD)
- Test outputbands on ASKAP oversampled/overlapping data after changing autobands logic is updated for the overlap regions (AD)
- Ocean loading: index by position rather than antenna name/code (Walter? + David Gordon)
- Want to circumvent 2-letter name overload, and ability to have two codes for one station
- Generally improve this experience.
- Documentation of polconvert on the wiki (Cormac)
- Awaiting test data from Parkes
- PolConvert single dish mode and document (GC)
- Update vex2xml for vex2 (GC)
- difx2fits: add option to split files at a given size (WB, CR)
- 20180907: Option implemented to split after specified number of jobs
- Update documentation on data formats (both for .v2d and mark5access) (WB, AD)
- difxcalc: update documentation & examples (AD)
- provide a new reference model in examples (AD)
- startdifx: document & improve the single machine case, make it use difxcalc by default (AD, Mike)
- Documentation
- Putting doxygen of mpifxcorr source on a webserver (link to this from the DiFX wiki if completed: Geoff)
- Investigate sampler stats monitoring (WB to dump out into .difx/ dir)
- subband distribution/parallelisation (with VDIF) (AD/CP)
- replumbing/refactoring vex2difx setup (DiFX 3)
- Add option to use band centres instead of band edges (shifted FFT) (WB) (DiFX 3?)
- Cleanly separate data format and data source (DiFX 3)
- Option for zero-padded FFT
- Extending cpumon to report current cpu speed and current power dissipation (WB)
- Generate some one page EOIs for the next call for SSEs on ADACS (AD)
- How to allocate nodes in a specific order under slurm (AD,CR,HR)
Low Priority
- Virtual trunk missing components (find out what is missing and update)
- Use extracted pulse cal to predict correlated tones and then remove from visibilities
- Investigate whether merging pcal plotting tools is possible (Jan)
- Testing architecture:
- testing IPP setup
- Jenkins → vlbi-mgr.ira.inaf.it:8080 - login using Google credentials. (Matteo, John)
- interlaced vdif issue - why does turning interlacing on not always fix the missing data? (GC, AD)
- Investigate: 10-node performance hit at WACO (WB)
- evlbi
- reconnection/long term drop-outs (CP)
- reliability with UDP data (CP)
- real-time delay/rate adjustment (CP)
- Store polynomial tau(l, m) generalized delay model in FITS (WB,JM)
- Document Algorithms
- monitoring, etc - (list of error codes) (AD)
- Move to a compressed text format for pcal files
- Modernize sniffer
- cleanup of DiFX error messages (appropriate verbosity level everywhere)
- DiFX2fits uv shifting in trunk (or general model replacement therapy) (JM/AD)
- Pulsar predictor support in “polyco”
- Get rid of printing to screen in mark5access (change to be called and printed from elsewhere)
- Get Eric to improve AIPS PCAL (multi tone)
- Fully complete now. Needs testing.
- Add DiFX diagnostics
- MPI parallel tracing of DiFX to look for bottlenecks
Fantasy wishlist
- Choose framework for unit testing. Implement test cases.
- Phased array
- Add per-IF phase polynomial in the calc file (and delay polynomial, too?)
- On-the-fly application of calibration (bandpasses etc) with appropriate accountability
- Space VLBI
- Baseline dependent averaging
- Auditing / refactoring the amplitude scaling
- Has relevance for CASA work
- Expand/improve difxmessage
- Two-stage FFT option for very large FFTs (e.g., when using 2 GHz wide bands; DiFX 3 at earliest)
difx/difx-todolist-2019.txt · Last modified: 2020/02/24 16:49 by adamdeller
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
