User Tools

Site Tools


difx:difx-todolist-2019

To-Do and Wish List

To be prioritised
Standing Items
  • New python scripts should consider python3 conformance
  • Documentathon with emphasis on inner workings (all)
  • Update the utils page to show what package supplies each tool (all)
    • Reconsider the approach…
  • Update the pdf documentation for currently undocumented tools, and keep updated (Adam, WFB)
High Priority
  • Determine why complex interlaced VDIF data is not fringing in DiFX-2.6.1 (CP, AD, JW)
  • Change outputbands logic to choose appropriate spectral regions in overlap cases (JW)
  • DiFX pcals with multiple datastreams (fix in difx2mark4). In progress. (HR, John)
  • Mark6 group handing plan (WFB, JJ, MW, JW, HR)
  • sanitycheckdifx: what files/issues need to be checked? Implement in vex2difx (CR)
    • polyco typos
    • binconfig typos
    • addPhaseCentre typo in RA, or Dec, or whatever
  • datasim: parallelize, document (Zheng)
  • mark6meta library: fix issue of reporting “module complete” when using incomplete Mark6 modules (HR)
  • DiFX library dependencies clean-up & move of functionality. Partially, perhaps fully, complete (WB,JW,MW)
  • Regression testing scripts (send new datasets to Matteo)
    • Plan: Matteo investigate Jenkins
    • A more sophisticated pipeline than just diffDiFX?
    • Target size: 10G
    • Look for VDIF, Mark5B, LBA, zoom …
  • vex2 support (relies on vex2 finalization and some support from field systems)
    • document almost complete
    • vex parser is almost complete
    • Support for $BITSTREAMS largely complete in vex2difx
    • Plan: finalise, test
  • Mark6 playback speed (locally high, but via network speeds are slower)
Medium Priority
  • Test outputbands on ASKAP oversampled/overlapping data after changing autobands logic is updated for the overlap regions (AD)
  • Ocean loading: index by position rather than antenna name/code (Walter? + David Gordon)
    • Want to circumvent 2-letter name overload, and ability to have two codes for one station
    • Generally improve this experience.
  • Documentation of polconvert on the wiki (Cormac)
    • Awaiting test data from Parkes
  • PolConvert single dish mode and document (GC)
  • Update vex2xml for vex2 (GC)
  • difx2fits: add option to split files at a given size (WB, CR)
    • 20180907: Option implemented to split after specified number of jobs
  • Update documentation on data formats (both for .v2d and mark5access) (WB, AD)
  • difxcalc: update documentation & examples (AD)
  • provide a new reference model in examples (AD)
  • startdifx: document & improve the single machine case, make it use difxcalc by default (AD, Mike)
  • Documentation
    • Putting doxygen of mpifxcorr source on a webserver (link to this from the DiFX wiki if completed: Geoff)
  • Investigate sampler stats monitoring (WB to dump out into .difx/ dir)
  • subband distribution/parallelisation (with VDIF) (AD/CP)
    • replumbing/refactoring vex2difx setup (DiFX 3)
  • Add option to use band centres instead of band edges (shifted FFT) (WB) (DiFX 3?)
  • Cleanly separate data format and data source (DiFX 3)
  • Option for zero-padded FFT
  • Extending cpumon to report current cpu speed and current power dissipation (WB)
  • Generate some one page EOIs for the next call for SSEs on ADACS (AD)
  • How to allocate nodes in a specific order under slurm (AD,CR,HR)
Low Priority
  • Virtual trunk missing components (find out what is missing and update)
  • Use extracted pulse cal to predict correlated tones and then remove from visibilities
  • Investigate whether merging pcal plotting tools is possible (Jan)
  • Testing architecture:
    • testing IPP setup
    • Jenkins → vlbi-mgr.ira.inaf.it:8080 - login using Google credentials. (Matteo, John)
  • interlaced vdif issue - why does turning interlacing on not always fix the missing data? (GC, AD)
  • Investigate: 10-node performance hit at WACO (WB)
  • evlbi
    • reconnection/long term drop-outs (CP)
    • reliability with UDP data (CP)
    • real-time delay/rate adjustment (CP)
  • Store polynomial tau(l, m) generalized delay model in FITS (WB,JM)
  • Document Algorithms
    • monitoring, etc - (list of error codes) (AD)
  • Move to a compressed text format for pcal files
  • Modernize sniffer
  • cleanup of DiFX error messages (appropriate verbosity level everywhere)
  • DiFX2fits uv shifting in trunk (or general model replacement therapy) (JM/AD)
  • Pulsar predictor support in “polyco”
  • Get rid of printing to screen in mark5access (change to be called and printed from elsewhere)
  • Get Eric to improve AIPS PCAL (multi tone)
    • Fully complete now. Needs testing.
  • Add DiFX diagnostics
  • MPI parallel tracing of DiFX to look for bottlenecks
Fantasy wishlist
  • Choose framework for unit testing. Implement test cases.
  • Phased array
  • Add per-IF phase polynomial in the calc file (and delay polynomial, too?)
  • On-the-fly application of calibration (bandpasses etc) with appropriate accountability
  • Space VLBI
    • Baseline dependent averaging
  • Auditing / refactoring the amplitude scaling
    • Has relevance for CASA work
  • Expand/improve difxmessage
  • Two-stage FFT option for very large FFTs (e.g., when using 2 GHz wide bands; DiFX 3 at earliest)
difx/difx-todolist-2019.txt · Last modified: 2019/11/19 11:31 by adamdeller