User Tools

Site Tools


difx:difx-todolist-2017

To-Do and Wish List

Before the end of the week
  • Clock units in vex2difx
    • progress: confirmed even newest FS vex library has no support for usec/sec units
    • progress: vex2difx currently behaves ligitimately – no changes required
    • plan: Ask Ed to add this. Status: currently, according to vex2 doc, a “time1/time2” unit must have time1 > time 2. Ed seems happy to relax this stipulation, perhaps even for vex1.5 (WFB)
  • Ensure that all files have been updated to reflect the new license. (Adam)
  • Testing architecture:
    • make link available
    • testing IPP setup
    • Jenkins → vlbi-mgr.ira.inaf.it:8080 - login using Google credentials. (Matteo, John)
    • Make test datasets useable again (v2d files, etc.)
  • Documentathon with emphasis on inner workings (all)
  • Single read of .input .calc .im .threads .binconfig .polyco (phased array) file by manager then distribute… (Adam)
  • polConvert=true in .v2d to avoid needing –force in vex2difx (Walter)
  • Mark6 workflow / file list / group handing plan (WFB, JJ, MW, JW)
  • vex2difx switch to allow calculation of autocrosspols from multi-datastream antennas (Adam)
  • Add checkmpifxcorr to startdifx (Geoff)
  • Update the utils page to show what package supplies each tool (all)
    • Reconsider the approach…
  • Update the pdf documentation for currently undocumented tools, and update pdf to version 2.5 (Adam, WFB)
  • Update the wiki documentation on how to check out just trunk and current releases (Cormac)
  • Update the wiki documentation on tagging guidelines (Adam)
  • Investigate whether merging pcal plotting tools is possible (Jan)
  • Documentation of polconvert on the wiki (Cormac)
    • Awaiting test data from Parkes
  • Put up a plan for how to arrange next meeting (Adam)
  • Investigate whether it is possible to have svn automatically emit a reminder to sign up to the mailing list when someone checks out code (Geoff)
    • Not trivial
    • Put in install-difx and difxbuild
  • Add mount type override to vex2difx if it doesn't already exist (Walter)
  • Contact Eric Greisen regarding support for 11,12,21,22 “stokes” type in FITS-IDI (Mark K.)
  • Get additional information regarding polconvert into FITS file history via difx2fits (Walter + Geoff)
    • Implement general .difx/*.history file absorption in difx2fits (Walter)
    • Generate .difx/polconvert.history file (Geoff)
  • Ocean loading: index by position rather than antenna name/code (Walter? + David Gordon)
    • Generally improve this experience
High Priority
  • Add fuse mounting support in mk5daemon/mk6daemon (MW)
  • Look into treating single thread VDIF as interlaced VDIF (WB)
    • Add warning to vex2difx to wean people off single-thread
  • Adding difxmessage support into fusemk6/vdifuse/jive5ab (JW)
  • dirlist: parse Mark6.slist file (Walter)
    • Mark W to confirm
  • Implement group/filelist handling for fuse-mounted Mark6 data (JJ, MW)
  • Regression testing scripts (send new datasets to Matteo)
    • Plan: Matteo investigate Jenkins
    • A more sophisticated pipeline than just diffDiFX?
    • Target size: 10G
    • Look for VDIF, Mark5B, LBA, zoom …
  • vex2 support (relies on vex2 finalization and some support from field systems)
    • document almost complete
    • vex parser is almost complete
    • Support for $BITSTREAMS largely complete in vex2difx
    • Plan: finalise, test
Medium Priority
  • GPU acceleration of the DiFX core code
    • Adam to rerun tests once new architecture.h.in is available
    • Chris to do some controlled (small-scale) benchmarking
  • Mount types in difxio/antenna_db, mainly to allow Nasmyth mounts to be handled properly (Walter)
  • Add DiFX diagnostics
  • Documentation
    • Putting doxygen of mpifxcorr source on a webserver (link to this from the DiFX wiki if completed: Geoff)
  • Maintain database of benchmarking results
    • at least the regression tests
    • Even better, using fake data also
  • visplot (CP, JS)
  • Investigate sampler stats monitoring (WB to dump out into .difx/ dir)
  • subband distribution/parallelisation (with VDIF) (AD/CP)
    • replumbing/refactoring vex2difx setup
  • Transition to using band centres instead of band edges (WB)
  • Cleanly separate data format and data source
  • Option for zero-padded FFT
Low Priority
  • evlbi
    • reconnection/long term drop-outs (CP)
    • reliability with UDP data (CP)
    • real-time delay/rate adjustment (CP)
  • Store polynomial tau(l, m) generalized delay model in FITS (WB,JM)
  • IF selection in difx2fits
  • addZoomFreq in the SETUP section (applies to all antennas)
  • Document Algorithms
    • monitoring, etc - (list of error codes) (AD)
  • Still produce one FITS file even when there was a clock change during the correlation
  • Refactoring the mpifxcorr/mk5daemon libraries and merging functionality
  • Move to a compressed text format for pcal files
  • Modernize sniffer
  • cleanup of DiFX error messages (appropriate verbosity level everywhere)
  • Shifted FT (moving channel boundary to edge of band) (AD)
  • DiFX2fits uv shifting in trunk (JM/AD)
  • Pulsar predictor support in “polyco”
  • Get rid of printing to screen in mark5access (change to be called and printed from elsewhere)
  • Get Eric to improve AIPS PCAL (multi tone) - underway now
Fantasy wishlist
  • Choose framework for unit testing. Implement test cases.
  • Phased array
  • Add per-IF phase polynomial in the calc file (and delay polynomial, too?)
  • On-the-fly application of calibration (bandpasses etc) with appropriate accountability
  • Space VLBI
    • Baseline dependent averaging
  • Auditing / refactoring the amplitude scaling
  • Restitching contiguous zoom bands into wider bands (e.g. for ALMA): Jan has an implementation of difx2difx!
  • Expand/improve difxmessage
  • Two-stage FFT option for very large FFTs (e.g. when using 2 GHz wide bands)
difx/difx-todolist-2017.txt · Last modified: 2018/09/06 00:38 by helgerottmann