[Back to Main Page]

LLProxy - HTTP Proxy Service
by Dennis Lang 2013

LLProxy v1.1
Aug 2013
Dennis Lang
Content:
  • Overview
  • Copyright and Licensing info
  • Install Server and enable Proxy
  • Monitor
  • Main c++ classes in Proxy
  • Proxy Data Flow
  • Links to other Proxy examples
  • Downloads:
  • LLProxy v1.1 source zip
  • (Windows/x32) LLProxy-srv.exe
  • (Windows/x32) pThread.dll
  • (Windows/x32) LLProxy-mon.exe
  • (Windows/x32) 3 executable parts zip'd

  • HTTP Proxy Overview

    I created the following portable HTTP proxy server with plans to log all of the http traffic on my system. Like most engineers I started by searching the web for reference material and existing bodies of work. The results of my web search are list at the bottom of this page. Most of the 'working' http proxies failed to support tunneling, which was a topic I was not familar with and discovered that a lot of data heavy web-sites like YouTube made use of this secondary connection to move large volumes of data.

    I eventually found the Proxy server by Sam King on the University of Illinois/NCSA web-site which had support for tunneling. I ported the code to Windows and tried to leave support for Unix. I did various code clean-up, such as porting C to C++ and removing #defines and using smart pointers to mamange memory.

    After I got the basic proxy working I added a shared memory portal so an external agent could monitor the proxy traffic. Next I added a front end to allow the proxy to run as a windows service. Lastly, I created a simple Windows Monitor UI to display the shared memory portal information.

    [To Top]


    Licensing and Copyright Information

    The body of the proxy server is from code found on:

    http://www.cs.uiuc.edu/homes/kingst/Research.html

    Developed by the University of Illinois/NCSA in the Research Group of Professor Sam King in the Department of Computer Science The University of Illinois at Urbana-Champaign
    ** (See full copyright notice in proxy files).

    The POSIX Threads Library for Win32 is provided by:


    Copyright(C) 1998 John E. Bossom
    Copyright(C) 1999,2005 Pthreads-win32 contributors

    Contact Email: rpj@callisto.canberra.edu.au

    The current list of contributors is contained in the file CONTRIBUTORS included with the source code distribution. The list can also be seen at the following World Wide Web location: http://sources.redhat.com/pthreads-win32/contributors.html ** (See full copyright notice in the pthread files).

    [To Top]


    Install

    To Install the Proxy server you can either run it via the command line, which is ideal for testing, or as a window service. Use -? to see the help banner (see image below).

    Next you want to enable the Proxy server. From Chrome go to Settings by first clicking on the top right icon. If you don't use Chrome click Control Panel to see alternate approach.

    An alternate approach is to use the Window's Control Panel and select the Internet Options and LAN settings to get to the same place.

    Enable the Proxy server by checking the toggle box and click on Advance to set the ports.

    The LLProxy server uses port 8000, set both HTTP and Secure to use this port.

    To disable the proxy, just uncheck the proxy box, you don't have to clear or erase the port configuration.

    [To Top]


    Running LLProxyMon to Monitor proxy traffic

    LLProxyMon is a simple C# windows program to display the HTTP URL traffic. Currently there is very little error handling and if you start it before starting the Proxy server it will fail to connect and will not retry.

    LLProxyMon uses a shared memory region and an event handler to handle the communication of the HTTP url traffic across the shared memory. This logic is similar to how the OutputDebugString message is handled.

    public static void MonitorThread()
    {
        const string sLLProxyMemName = "Global\\llproxy";
        const int sMemSize = 8192;
        SharedMemory m_sharedMemory;
        IntPtr m_MsgEvent = IntPtr.Zero;
    
        try
        {
            m_sharedMemory = new SharedMemory(sLLProxyMemName, sMemSize);
        }
        catch (Exception ex)
        {
            SetStatus(MonitorStatus.eExit, ex.Message);
            return;
        }
    
        // Create the event for slot 'DBWIN_BUFFER_READY'
        const string sEventName1 = "llproxyEvent1";
        WinEvent.SECURITY_ATTRIBUTES sa = new WinEvent.SECURITY_ATTRIBUTES();
        m_MsgEvent = WinEvent.CreateEvent(ref sa, false, false, sEventName1);
        if (m_MsgEvent == IntPtr.Zero)
        {
            SetStatus(MonitorStatus.eExit, "Failed to create event 'llproxyEvent1'");
            return;
        }
    

    The shared memory has a small header of 4 unsigned integers

    struct MemHeader
    {
        uint    memSize;
        uint    hdrSize;
        uint    postOff;    // end of current message.
        uint    postSeq;
    };
    
    The memSize is initialized with the size of the shared memory region and the hdrSize with the size of the MemHeader structure.

    As the Proxy Services posts messages it copies it into the shared memory at the postOff position. Once the copy has completed the postOff value is moved to the end of the new message and the postSeq is incremented and the event handler is set.

    bool ShareMsg::PostMsg(uint threadId, const std::string& host, const HTTP& http, int respondsLen)
    {
        bool status = false;
        // WinSharedMem m_sharedMem;
    
        std::ostringstream sout;
        const char  sSep[] = "|";
        sout << GetTimeString() << sSep << threadId << sSep << host << sSep << http.GetUrl() << sSep << respondsLen << "\n";
        std::string msg = sout.str();
    
        uint msgLen = msg.length();
        if (msgLen < sMemMsgSize)
        {
            status = true;
            uint cpyLen = min(sMemMsgSize - m_memHdrPtr->postOff, msgLen);
    
            memcpy(m_memPtr + m_memHdrPtr->postOff, msg.c_str(), cpyLen);
            m_memHdrPtr->postOff = (m_memHdrPtr->postOff + cpyLen) % sMemMsgSize;
            msgLen -= cpyLen;
    
            if (msgLen != 0)
            {
                memcpy(m_memPtr + m_memHdrPtr->postOff, msg.c_str() + cpyLen, msgLen);
                m_memHdrPtr->postOff += msgLen;
            }
    
            m_memHdrPtr->postSeq++;
            SetEvent(m_eventHnd);
        }
    
        return status;
    }
    
    As the messages fill the shared memory region, they wrap around and over write previous messages. The share memory is not locked by any critical sections or mutexes and has no performance impact on the Proxy Server. If the Monitor program is running too slowly or does not use a separate thread to monitor the shared memory it could get behind and produce garbled messages. The sequence number is one way for the monitor to detect that it lost a message by comparing its sequence number with that in the shared memory.

    [To Top]


    Main classes in Proxy Service


    Proxy data flow

    The proxy class is very simple, it just waits for a HTTP message which causes a TCP accept to complete and spawns a thread to parse the message and post the URL information to the share memory.
    // ------------------------------------------------------------------------------------------------
    void LLProxy::StartServer(unsigned int port)
    {
        DebugMsg() << "Starting server on port " << port << std::endl;
    
        LLServerSocket server(port);
        m_abort = false;
    
        while (!m_abort)
        {
            ThreadInfo& threadInfo = NextFreeInfo();
            server.Accept(threadInfo.clientSock);
            StartClient(threadInfo);
        }
    }
    
    // ------------------------------------------------------------------------------------------------
    void LLProxy::StartClient(ThreadInfo& threadInfo)
    {
        int ret = pthread_create(&threadInfo.tid, NULL, ClientThread, &threadInfo);
        assert(ret == 0);
        ret = pthread_detach(threadInfo.tid);
        assert(ret == 0);
    }
    
    // ------------------------------------------------------------------------------------------------
    void LLProxy::RunClient(ThreadInfo& threadInfo)
    {
        HttpRequest request(threadInfo.clientSock);
        if (!request.ReadRequest())
        {
            // cout << "Did not read request" << endl;
        }
        else
        {
            int msgLen = HttpRelay::GetRelay().GetHttpResponse(request.GetHost(), request.GetRequest(),
                    request.GetUrl(), threadInfo.clientSock, request.isConnect());
            ShareMsg::Instance().PostMsg(GetCurrentThreadId(), request.GetHost(), request.GetHttp(), msgLen);
        }
    
        FreeInfo(threadInfo);
        DebugMsg() << GetCurrentThreadId() << " Done\n";
    }
    

    The HTTP parsing and processing is handled by the University of Illinois/NCSA code written by Sam King. The HTTP classes complete the request and returns the response. The top level code takes the Host and URL string and posts it to the shared memory.

    [To Top]


    Web Links to other HTTP Proxy examples

    C/C++ Proxy Servers

  • HTTP Tunneling (HTTP Proxy Socket Client) by Akash Kava on April 21st, 2003 on Codeguru

  • A lightwegith HTTP proxy written in C++, MIT license, c++, on code.google.com

  • A small and simple HTTP proxy server by ankurdhama, 8/24/2003, on Planet-Source-Code.com

  • Cross-Platform and small C++ HTTP api. This library currently supports:

    • Authentication: Basic, digest and NTLM.
    • Cookies: Manual and automatic cookie handling, using internal btree.
    • Redirection: The library is currently able to handle HTTP redirects (301, 302 and 303)
    • HTTP Proxy: The API allows to create multiple HTTP proxy instances without adding new code.
    • HTTPS Proxy: By design, the HTTPCore API allows the interception of SSL traffic.
    • Bandwidth: The used bandwidth among other HTTP limits can be stablished.
    • Transfer Encoding: Automatic chunk encoding and decoding.
    • Content Encoding: Supports gzip and deflate.
    • Persistent connections: HTTP/1.0 and HTTP/1.1 support with persistent connections.
    • Protocols: Supports both HTTP and HTTPS protocols.
    • Multithreading: Fully thread safe api.
    • Callbacks: Integrated with an internal callback system that allows flexible data manipulation by third part plugins.

    There are also some new features we are currently working on:

    • IPV6: Ipv6 support will be finished soon.
    • UNICODE Support: Working under win32.

    Check the API reference manual for information about how does Fhscan HTTP API works.

  • A simple implementation of an HTTP proxy server and client in C++ using sockets and pthreads. Basic server-side caching is also implemented. Code: C++ (Unix), Date: 2012

  • Reverse C++ TCP Proxy Server, 2007 Arash Partow
    The C++ TCP Proxy server is a simple and high performance utility using the ASIO networking library, for proxying (tunneling or redirecting) connections from external clients to a designated server. The TCP Proxy server can be used to easily and efficiently:

    • Limit the number of client connections to the server
    • Load balance client connections between multiple server instances
    • Provide IP or connection time based filtering and access control mechanisms

  • A multi thread HTTP proxy, 2011, uses University of Illinois/NCSA developed by Sam King
    This is a very basic HTTP proxy. Currently it will create a new thread for each new client connection and use blocking I/O calls to get the request from the browser and then get a reply from the server.
  • Non-C++ Proxy servers

  • This is an RPG IV service program that uses socket calls to implement the HTTP 1.1 protocol. The HTTP protocol is the data transfer protocol that is used to transfer documents over the World Wide Web.

  • A proxy for APIs to allow rate-limiting

  • A full-featured http proxy for node.js

    Features

    • Reverse proxies incoming http.ServerRequest streams
    • Can be used as a CommonJS module in node.js
    • Uses event buffering to support application latency in proxied requests
    • Reverse or Forward Proxy based on simple JSON-based configuration
    • Supports WebSockets
    • Supports HTTPS
    • Minimal request overhead and latency
    • Full suite of functional tests
    • Battled-hardened through production usage @ nodejitsu.com
    • Written entirely in Javascript
    • Easy to use API

  • Generic HTTP/HTTPS proxy component, currently without caching support, based on work of Wilfried Mestdagh. Requires ICS to compile.
    Code: Written in Pascal
    Date: 3/17/2004

  • Commerical Proxy Servers

  • Fiddler WEB Debugging Proxy, FREE

  • Charles - WEB Debugging Proxy Application for Windows, Mac OS and Linuy, $$$
  • [To Top]