difx:difx-todolist-2018
To-Do and Wish List
Standing Items
- New python scripts should consider python3 conformance
- Documentathon with emphasis on inner workings (all)
- Update the utils page to show what package supplies each tool (all)
- Reconsider the approach…
- Update the pdf documentation for currently undocumented tools, and keep updated (Adam, WFB)
Before the end of the week
difx2fits: print why splitting files (WB)genmachines: add validation for group and module completenes of Mark6 media (HR)
High Priority
Share vex2xml with difx trunk (GC)- DiFX pcals with multiple datastreams (fix in difx2mark4). In progress. (HR, John)
- Mark6 group handing plan (WFB, JJ, MW, JW, HR)
Mergebands in vex2difx / mpifxcorr (bringing in difx2difx functionality into the real-time part of the correlator) (JW)- sanitycheckdifx: what files/issues need to be checked? Implement in vex2difx (CR)
- polyco typos
- binconfig typos
- addPhaseCentre typo in RA, or Dec, or whatever
- datasim: parallelize, document (Zheng)
difx2mark4: fix segfault issue (HR, GC)- mark6meta library: fix issue of reporting “module complete” when using incomplete Mark6 modules (HR)
CODIF branch merge (AD, CP)Tag 2.5.3 release (GC)- DiFX library dependencies clean-up & move of functionality. Partially, perhaps fully, complete (WB,JW,MW)
Adding difxmessage support into <del>fusemk6/vdifuse/jive5ab (JW) </del>Implement group/filelist handling for fuse-mounted Mark6 data (JJ, MW). Superseded- Regression testing scripts (send new datasets to Matteo)
- Plan: Matteo investigate Jenkins
- A more sophisticated pipeline than just diffDiFX?
- Target size: 10G
- Look for VDIF, Mark5B, LBA, zoom …
- vex2 support (relies on vex2 finalization and some support from field systems)
- document almost complete
- vex parser is almost complete
- Support for $BITSTREAMS largely complete in vex2difx
- Plan: finalise, test
Medium Priority
- Ocean loading: index by position rather than antenna name/code (Walter? + David Gordon)
- Want to circumvent 2-letter name overload, and ability to have two codes for one station
- Generally improve this experience.
- Documentation of polconvert on the wiki (Cormac)
- Awaiting test data from Parkes
Single read of .input .calc .im .threads .binconfig .polyco (phased array) file by manager then distribute… (Adam, JW)Clock units in vex2difx- progress: confirmed even newest FS vex library has no support for usec/sec units
- progress: vex2difx currently behaves ligitimately – no changes required
- Status: currently, according to vex2 doc, a “time1/time2” unit must have time1 > time 2. Ed seems happy to relax this stipulation. This will be implemented in vex2 - check when this is out. (WFB)
- Update vex2xml for vex2 (GC)
importfitsidi: what's missing ? (MK)- other CASA VLBI issues (MK)
- difx2fits: add option to split files at a given size (WB, CR)
- 20180907: Option implemented to split after specified number of jobs
- Update documentation on data formats (both for .v2d and mark5access) (WB, AD)
- difxcalc: update documentation & examples (AD)
- provide a new reference model in examples (AD)
- startdifx: document & improve the single machine case, make it use difxcalc by default (AD, Mike)
- Documentation
- Putting doxygen of mpifxcorr source on a webserver (link to this from the DiFX wiki if completed: Geoff)
visplot (CP, JS). Superseded.- Investigate sampler stats monitoring (WB to dump out into .difx/ dir)
- subband distribution/parallelisation (with VDIF) (AD/CP)
- replumbing/refactoring vex2difx setup (DiFX 3)
- Add option to use band centres instead of band edges (shifted FFT) (WB) (DiFX 3?)
- Cleanly separate data format and data source (DiFX 3)
- Option for zero-padded FFT
Low Priority
- Use extracted pulse cal to predict correlated tones and then remove from visibilities
- Investigate whether merging pcal plotting tools is possible (Jan)
- Testing architecture:
- testing IPP setup
- Jenkins → vlbi-mgr.ira.inaf.it:8080 - login using Google credentials. (Matteo, John)
- interlaced vdif issue - why does turning interlacing on not always fix the missing data? (GC, AD)
- Investigate: 10-node performance hit at WACO (WB)
- evlbi
- reconnection/long term drop-outs (CP)
- reliability with UDP data (CP)
- real-time delay/rate adjustment (CP)
- Store polynomial tau(l, m) generalized delay model in FITS (WB,JM)
- Document Algorithms
- monitoring, etc - (list of error codes) (AD)
Still produce one FITS file even when there was a clock change during the correlation (WB; check to see how AIPS handles merging of FITS files w/ different clocks)- Move to a compressed text format for pcal files
- Modernize sniffer
- cleanup of DiFX error messages (appropriate verbosity level everywhere)
- DiFX2fits uv shifting in trunk (or general model replacement therapy) (JM/AD)
- Pulsar predictor support in “polyco”
- Get rid of printing to screen in mark5access (change to be called and printed from elsewhere)
- Get Eric to improve AIPS PCAL (multi tone)
- Fully complete now. Needs testing.
- Add DiFX diagnostics
Fantasy wishlist
- Choose framework for unit testing. Implement test cases.
- Phased array
- Add per-IF phase polynomial in the calc file (and delay polynomial, too?)
- On-the-fly application of calibration (bandpasses etc) with appropriate accountability
- Space VLBI
- Baseline dependent averaging
- Auditing / refactoring the amplitude scaling
- Has relevance for CASA work
- Expand/improve difxmessage
- Two-stage FFT option for very large FFTs (e.g., when using 2 GHz wide bands; DiFX 3 at earliest)
difx/difx-todolist-2018.txt · Last modified: 2019/11/15 12:12 by cormac
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
