User Tools

Site Tools


difx:difx-todolist-2019

This is an old revision of the document!


To-Do and Wish List

Standing Items
  • New python scripts should consider python3 conformance
  • Documentathon with emphasis on inner workings (all)
  • Update the utils page to show what package supplies each tool (all)
    • Reconsider the approach…
  • Update the pdf documentation for currently undocumented tools, and keep updated (Adam, WFB)
Before the end of the week
  • difx2fits: print why splitting files (WB)
  • genmachines: add validation for group and module completenes of Mark6 media (HR)
High Priority
  • Share vex2xml with difx trunk (GC)
  • DiFX pcals with multiple datastreams (fix in difx2mark4). In progress. (HR, John)
  • Mark6 group handing plan (WFB, JJ, MW, JW, HR)
  • Mergebands in vex2difx / mpifxcorr (bringing in difx2difx functionality into the real-time part of the correlator) (JW)
  • sanitycheckdifx: what files/issues need to be checked? Implement in vex2difx (CR)
    • polyco typos
    • binconfig typos
    • addPhaseCentre typo in RA, or Dec, or whatever
  • datasim: parallelize, document (Zheng)
  • difx2mark4: fix segfault issue (HR, GC)
  • mark6meta library: fix issue of reporting “module complete” when using incomplete Mark6 modules (HR)
  • CODIF branch merge (AD, CP)
  • Tag 2.5.3 release (GC)
  • DiFX library dependencies clean-up & move of functionality. Partially, perhaps fully, complete (WB,JW,MW)
  • Adding difxmessage support into <del>fusemk6/vdifuse/jive5ab (JW) </del>
  • Implement group/filelist handling for fuse-mounted Mark6 data (JJ, MW). Superseded
  • Regression testing scripts (send new datasets to Matteo)
    • Plan: Matteo investigate Jenkins
    • A more sophisticated pipeline than just diffDiFX?
    • Target size: 10G
    • Look for VDIF, Mark5B, LBA, zoom …
  • vex2 support (relies on vex2 finalization and some support from field systems)
    • document almost complete
    • vex parser is almost complete
    • Support for $BITSTREAMS largely complete in vex2difx
    • Plan: finalise, test
Medium Priority
  • Ocean loading: index by position rather than antenna name/code (Walter? + David Gordon)
    • Want to circumvent 2-letter name overload, and ability to have two codes for one station
    • Generally improve this experience.
  • Documentation of polconvert on the wiki (Cormac)
    • Awaiting test data from Parkes
  • Single read of .input .calc .im .threads .binconfig .polyco (phased array) file by manager then distribute… (Adam, JW)
  • Clock units in vex2difx
    • progress: confirmed even newest FS vex library has no support for usec/sec units
    • progress: vex2difx currently behaves ligitimately – no changes required
    • Status: currently, according to vex2 doc, a “time1/time2” unit must have time1 > time 2. Ed seems happy to relax this stipulation. This will be implemented in vex2 - check when this is out. (WFB)
  • Update vex2xml for vex2 (GC)
  • importfitsidi: what's missing ? (MK)
    • other CASA VLBI issues (MK)
  • difx2fits: add option to split files at a given size (WB, CR)
    • 20180907: Option implemented to split after specified number of jobs
  • Update documentation on data formats (both for .v2d and mark5access) (WB, AD)
  • difxcalc: update documentation & examples (AD)
  • provide a new reference model in examples (AD)
  • startdifx: document & improve the single machine case, make it use difxcalc by default (AD, Mike)
  • Documentation
    • Putting doxygen of mpifxcorr source on a webserver (link to this from the DiFX wiki if completed: Geoff)
  • visplot (CP, JS). Superseded.
  • Investigate sampler stats monitoring (WB to dump out into .difx/ dir)
  • subband distribution/parallelisation (with VDIF) (AD/CP)
    • replumbing/refactoring vex2difx setup (DiFX 3)
  • Add option to use band centres instead of band edges (shifted FFT) (WB) (DiFX 3?)
  • Cleanly separate data format and data source (DiFX 3)
  • Option for zero-padded FFT
Low Priority
  • Use extracted pulse cal to predict correlated tones and then remove from visibilities
  • Investigate whether merging pcal plotting tools is possible (Jan)
  • Testing architecture:
    • testing IPP setup
    • Jenkins → vlbi-mgr.ira.inaf.it:8080 - login using Google credentials. (Matteo, John)
  • interlaced vdif issue - why does turning interlacing on not always fix the missing data? (GC, AD)
  • Investigate: 10-node performance hit at WACO (WB)
  • evlbi
    • reconnection/long term drop-outs (CP)
    • reliability with UDP data (CP)
    • real-time delay/rate adjustment (CP)
  • Store polynomial tau(l, m) generalized delay model in FITS (WB,JM)
  • Document Algorithms
    • monitoring, etc - (list of error codes) (AD)
  • Still produce one FITS file even when there was a clock change during the correlation (WB; check to see how AIPS handles merging of FITS files w/ different clocks)
  • Move to a compressed text format for pcal files
  • Modernize sniffer
  • cleanup of DiFX error messages (appropriate verbosity level everywhere)
  • DiFX2fits uv shifting in trunk (or general model replacement therapy) (JM/AD)
  • Pulsar predictor support in “polyco”
  • Get rid of printing to screen in mark5access (change to be called and printed from elsewhere)
  • Get Eric to improve AIPS PCAL (multi tone)
    • Fully complete now. Needs testing.
  • Add DiFX diagnostics
Fantasy wishlist
  • Choose framework for unit testing. Implement test cases.
  • Phased array
  • Add per-IF phase polynomial in the calc file (and delay polynomial, too?)
  • On-the-fly application of calibration (bandpasses etc) with appropriate accountability
  • Space VLBI
    • Baseline dependent averaging
  • Auditing / refactoring the amplitude scaling
    • Has relevance for CASA work
  • Expand/improve difxmessage
  • Two-stage FFT option for very large FFTs (e.g., when using 2 GHz wide bands; DiFX 3 at earliest)
difx/difx-todolist-2019.1573780564.txt.gz · Last modified: 2019/11/15 12:16 by cormac