Package net.i2p.util

Class EepGet

    • Field Detail

      • _log

        protected final Log _log

      • _shouldProxy

        protected final boolean _shouldProxy

      • _proxyHost

        protected final String _proxyHost

      • _proxyPort

        protected final int _proxyPort

      • _numRetries

        protected final int _numRetries

      • _outputFile

        protected final String _outputFile

      • _url

        protected final String _url
        url we were asked to fetch

      • _actualURL

        protected String _actualURL
        the URL we actually fetch from (may differ from the _url in case of redirect)

      • _extraHeaders

        protected List<String> _extraHeaders

      • _keepFetching

        protected boolean _keepFetching

      • _proxy

        protected Socket _proxy

      • _alreadyTransferred

        protected long _alreadyTransferred

      • _bytesTransferred

        protected long _bytesTransferred

      • _bytesRemaining

        protected long _bytesRemaining

      • _currentAttempt

        protected int _currentAttempt

      • _responseCode

        protected int _responseCode

      • _responseText

        protected String _responseText

      • _shouldWriteErrorToOutput

        protected boolean _shouldWriteErrorToOutput

      • _etag

        protected String _etag

      • _lastModified

        protected String _lastModified

      • _etagOrig

        protected final String _etagOrig

      • _lastModifiedOrig

        protected final String _lastModifiedOrig

      • _encodingChunked

        protected boolean _encodingChunked

      • _notModified

        protected boolean _notModified

      • _contentType

        protected String _contentType

      • _transferFailed

        protected boolean _transferFailed

      • _aborted

        protected volatile boolean _aborted

      • _fetchHeaderTimeout

        protected int _fetchHeaderTimeout

      • _fetchTotalTimeout

        protected int _fetchTotalTimeout

      • _fetchInactivityTimeout

        protected int _fetchInactivityTimeout

      • _redirects

        protected int _redirects

      • _redirectLocation

        protected String _redirectLocation

      • _isGzippedResponse

        protected boolean _isGzippedResponse

      • _decompressException

        protected IOException _decompressException

      • USER_AGENT

        protected static final String USER_AGENT
        this will be replaced by the HTTP Proxy if we are using it
        See Also:
        Constant Field Values

      • MAX_COMPLETE_FAILS

        protected static final int MAX_COMPLETE_FAILS
        maximum times to try without getting any data at all, even if numRetries is higher @since 0.7.14
        See Also:
        Constant Field Values

    • Constructor Detail

      • EepGet

        public EepGet​(I2PAppContext ctx,
              boolean shouldProxy,
              String proxyHost,
              int proxyPort,
              int numRetries,
              String outputFile,
              String url,
              boolean allowCaching,
              String etag)

      • EepGet

        public EepGet​(I2PAppContext ctx,
              boolean shouldProxy,
              String proxyHost,
              int proxyPort,
              int numRetries,
              long minSize,
              long maxSize,
              String outputFile,
              OutputStream outputStream,
              String url,
              boolean allowCaching,
              String etag,
              String postData)
        Parameters:
        outputFile - ignored if outputStream is non-null
        outputStream - takes precedence over outputFile

      • EepGet

        public EepGet​(I2PAppContext ctx,
              boolean shouldProxy,
              String proxyHost,
              int proxyPort,
              int numRetries,
              long minSize,
              long maxSize,
              String outputFile,
              OutputStream outputStream,
              String url,
              boolean allowCaching,
              String etag,
              String lastModified,
              String postData)
        Parameters:
        outputFile - ignored if outputStream is non-null
        outputStream - takes precedence over outputFile

    • Method Detail

      • main

        public static void main​(String[] args)
        EepGet [-p 127.0.0.1:4444] [-n #retries] [-e etag] [-o outputFile] [-m markSize lineLen] url As of 0.9.45, supports https and redirect to https

      • suggestName

        public static String suggestName​(String url)
        Parse URL for a viable filename.
        Parameters:
        url - a URL giving the location of an online resource
        Returns:
        a filename to save the resource as on local filesystem

      • stopFetching

        public void stopFetching()

      • fetch

        public boolean fetch()
        Blocking fetch, returning true if the URL was retrieved, false if all retries failed. Header timeout default 45 sec, total timeout default none, inactivity timeout default 60 sec.
        Returns:
        success

      • fetch

        public boolean fetch​(long fetchHeaderTimeout)
        Blocking fetch, timing out individual attempts if the HTTP response headers don't come back in the time given. If the timeout is zero or less, this will wait indefinitely. Total timeout default none, inactivity timeout default 60 sec.
        Returns:
        success

      • fetch

        public boolean fetch​(long fetchHeaderTimeout,
                     long totalTimeout,
                     long inactivityTimeout)
        Blocking fetch.
        Parameters:
        fetchHeaderTimeout - <= 0 for none (proxy will timeout if none, none isn't recommended if no proxy)
        totalTimeout - <= 0 for default none
        inactivityTimeout - <= 0 for default 60 sec
        Returns:
        success

      • doFetch

        protected void doFetch​(SocketTimeout timeout)
                throws IOException
        This reads the response to a single fetch. Call after sendRequest()
        Parameters:
        timeout - may be null
        Throws:
        IOException

      • shouldReadBody

        protected boolean shouldReadBody()
        Should we read the body of the response?
        Returns:
        true always, overridden in EepHead
        Since:
        0.9.50

      • readChunkLength

        protected long readChunkLength()
                        throws IOException
        TODO this does not skip over chunk extensions (RFC 2616 sec. 3.6.1)
        Throws:
        IOException

      • getETag

        public String getETag()
        After fetch, the received value from the server, or null if none. Before fetch, and after some errors, may be the value passed in the constructor.

      • getLastModified

        public String getLastModified()
        After fetch, the received value from the server, or null if none. Before fetch, and after some errors, may be the value passed in the constructor.

      • getNotModified

        public boolean getNotModified()
        Returns:
        true if the server returned 304

      • getContentType

        public String getContentType()
        After fetch, the received value from the server, or null if none.

      • getStatusCode

        public int getStatusCode()
        The server response (200, etc).
        Returns:
        -1 if invalid, or if the proxy never responded, or if no proxy was used and the server never responded. If a non-proxied request partially succeeded (for example a redirect followed by a fail, or a partial fetch followed by a fail), this will be the last status code received. Note that fetch() may return false even if this returns 200.
        Since:
        0.8.8

      • getStatusText

        public String getStatusText()
        The server text ("OK", "Not Found", etc). Note that the text may contain % encoding.
        Returns:
        null if invalid, or if the proxy never responded, or if no proxy was used and the server never responded. If a non-proxied request partially succeeded (for example a redirect followed by a fail, or a partial fetch followed by a fail), this will be the last status code received. Note that fetch() may return false even if this returns "OK".
        Since:
        0.9.9

      • setWriteErrorToOutput

        public void setWriteErrorToOutput()
        If called (before calling fetch()), data from the server or proxy will be written to the output file or stream even on an error response code (4xx, 5xx, etc). The error data will only be written if no previous data was written on an earlier try. Caller must of course check getStatusCode() or the fetch() return value.
        Since:
        0.8.8

      • addHeader

        public void addHeader​(String name,
                      String value)
        Add an extra header to the request. Must be called before fetch(). Not supported by EepHead. As of 0.9.10, If name is User-Agent, this will replace the default User-Agent header. As of 0.9.14, If name is If-None-Match or If-Modified-Since, this will replace the etag or last-modified value given in the constructor. Note that headers may be subsequently modified or removed in the I2PTunnel HTTP Client proxy. In proxied SSLEepGet, these headers are sent to the remote server, NOT the proxy.
        Since:
        0.8.8

      • addAuthorization

        public void addAuthorization​(String userName,
                             String password)
        Add basic authorization header for the proxy. Only added if the request is going through a proxy. Must be called before fetch().
        Since:
        0.8.9

      • parseAuthArgs

        public static Map<String,​String> parseAuthArgs​(String args)
        Parse the args in an authentication header. Modified from LoadClientAppsJob. All keys are mapped to lower case. Double quotes around values are stripped. Ref: RFC 2617 Public for I2PTunnelHTTPClientBase; use outside of tree at own risk, subject to change or removal
        Parameters:
        args - non-null, starting after "Digest " or "Basic "
        Since:
        0.9.4, moved from I2PTunnelHTTPClientBase in 0.9.12