Archived data at SPDF are available at https://spdf.gsfc.nasa.gov/pub/ and also via FTP with TLS encryption at ftp://spdf.gsfc.nasa.gov/pub/ with specific FTP client tools.
As part of a general Federal policy that requires all network communications to be encrypted, SPDF has phased out access to unencrypted FTP services as of July 31, 2019. After the phaseout, SPDF will still support TLS-encrypted FTP, aka, FTPS: https://en.wikipedia.org/wiki/FTPS. Note that FTPS should not be confused with SFTP, which, despite its similar name, is not related to traditional FTP. Specifically SPDF will support explicit, passive-mode FTPS connections. Implicit FTPS is not supported, nor is active mode FTP.
What You Can Do:
We strongly urge users to check for any FTP dependencies in their scripts and processes that access SPDF. If you have questions or encounter problems, email SPDF Support.
Implications of This Transition for the User
Browsers: Most web browsers (Chrome, Firefox, etc.) do not support FTPS; however, users should normally be able to replace FTP URLs with HTTPS-based URLs, e.g.,
ftp://spdf.gsfc.nasa.gov/pub/data -> https://spdf.gsfc.nasa.gov/pub/data
ftp://cdaweb.gsfc.nasa.gov/pub/data -> https://cdaweb.gsfc.nasa.gov/pub/data
Command line tools:
Besides browsers, the biggest problem with moving to FTPS is that the standard old ftp client (inetutils) does not support FTPS. The following tools appear to work with FTPS. Perhaps the simplest replacement for command line FTP is LFTP below.
LFTP (“yum install lftp” or similar on Linux)
Perhaps linking lftp to ftp in /usr/bin will be sufficient, especially for old scripts.
Use ftp:// and NOT ftps:// URLs
lftp -e ‘set dns:order inet inet6’ -u anonymous,lftp@ spdf.gsfc.nasa.gov/pub/
CURL needs to be a recent version, >= 7.11, preferably > 7.54
Use ftp:// and NOT ftps:// URLs
curl —ssl —ftp-ssl-ccc —ftp-ssl-ccc-mode active ftp://pwgdata.gsfc.nasa.gov/ or ftp://spdf.gsfc.nasa.gov/
use: -kvvv to print debug output
Perl with Net::FTP (requires recent versions of the Net::FTP and IO::Socket::SSL Perl libraries)
Use Net::FTP;
my $ftp = Net::FTP->new($host, Passive => 1);
$ftp->starttls ();
$ftp->login(“anonymous”,‘-anonymous@‘)
$ftp->cwd(“/pub”)
$ftp->get(“that.file”)
$ftp->quit;
WGET (at least V1.17 November 2015)
Use ftps:// and NOT ftp:// URLs
wget “ftps://FTPSERVER/PATH”
User-friendly programs with FTPS support:
Filezilla (not recommended due to inappropriate bundling)
Be sure to change settings on installation: select Advanced and turn off browser changes and skip other installs. Select FTP for Protocol and select either ‘Use explicit FTP over TLS if available’ or ‘Required explicit FTP over TLS’
Cyberduck (select FTP-SSL (Explicit AUTH TLS) or use ftps:// URL)
Other Software:
Scripting:
Existing scripts are likely to require some modification to handle FTPS. In some cases, the change to HTTPS URLs noted above might be all that is needed, just switch to the secure HTTP protocol. If that is not feasible or desirable, many tools, e.g. recent versions of WGET (at least V1.17 November 2015) and CURL support FTPS, but generally these will need to be modified to specify that an SSL session is to be used. Note that the URL that is used with CURL will usually still be specified as ftp:// even after the transition to FTPS, but WGET seems to use ftps://.
Additional notes:
PYTHON FTP_TLS library for Python 3
from ftplib import FTP_TLS
ftp=FTP_TLS(FTPSERVER)
ftp.login()
ftp.dir()
ftp.cwd(PATH)
ftp.retrbinary(‘RETR FILENAME’, open(‘FILENAME’, ‘wb’).write)
ftp.quit()
kermit FTPSERVER
set auth tls debug on
set auth tls verbose on
set ftp authtype tls
set ftp debug on
set auth tls verify no
VMS notes
http://www.process.com/docs/multinet5_5/users_guide/appendix_b.html
$ ftp /AUTHENTICATE=TLS /verbose/u=ftp/pass=guest FTPSERVER
automatically sends CCC add commands to SYS$LOGIN:FTP.INIT
TCPware: $ ftp /verb/tls/u=ftp/pass=guest FTPSERVER
$ curl “—ssl-reqd -k -vvv” “ftp://FTPSERVER”
Kermit might also work.
**************** Programs that do NOT support TLS encryption *********************
old ftp client (inetutils) ncftp tnftp (BSD ftp)
**************** FTP-SSL upload *********************
For the few groups that need to upload files to a FTP server, some possibilities are below.
LFTP (“yum install lftp” or similar on Linux)
Perhaps linking lftp to ftp in /usr/bin will be sufficient, especially for old scripts.
lftp ‘set dns:order inet inet6’ -e ‘set ftp:ssl-allow true’ -e ‘set ssl:verify-certificate false’ -u ACCOUNT SERVER
You can move the switches to the ~/.lftprc file, but they will affect all lftp connections:
set dns:order inet inet6
set ftp:ssl-allow true
# set ftp:ssl-force true
set ssl:verify-certificate false
You can save the password in the ~/.netrc file:
machine SERVER login MYACCOUNT MYPASSWORD
curl —ftp-ssl -T “FILE.TXT” -k -u “MYUSER:MYPASSWORD” “ftp://SERVER”
other options:
add “-n” to read .netrc file for username and password
-w size_upload # will return bytes uploaded
-w ssl_verify_result
multiple files can be uploaded with -T “{file1,file2}”
move files: “-Q rename source target”
example to ftp a bunch of like named files to an ftps server, where the machine, username and password are defined in a .netrc file:
for i in wi\_\*2016\*\_v02.cdf; do
curl --ftp-ssl --netrc-file $HOME/.netrc -T ${i} -k -w size\_upload ftp://machine.gsfc.nasa.gov ;
donewput −−force-tls
Python FTP_TLS library upload
ftp.storbinary(‘STOR myfile.txt’, open(‘myfile.txt’, ‘rb’).read)