Niko/forwardproxy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Initial commit

Browse source
This commit is contained in:
Sergey Frolov 2017-07-24 12:05:47 -04:00 • committed by Sergey Frolov
commit 9e92549830
18 changed files with 2039 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
.travis.yml Normal file
View file

@ -0,0 +1,11 @@
language: go
go:
- 1.8.x
# - 1.9.x
dist: trusty
script:
- go test .
- unformatted="$(gofmt -l .)"; ! [ -z "${unformatted}" ] && echo "${unformatted}" && exit 1

23
CONTRIBUTING.md Normal file
View file

@ -0,0 +1,23 @@
# How to Contribute
We'd love to accept your patches and contributions to this project. There are
just a few small guidelines you need to follow.
## Contributor License Agreement
Contributions to this project must be accompanied by a Contributor License
Agreement. You (or your employer) retain the copyright to your contribution,
this simply gives us permission to use and redistribute your contributions as
part of the project. Head over to <https://cla.developers.google.com/> to see
your current agreements on file or to sign a new one.
You generally only need to submit a CLA once, so if you've already submitted one
(even if it was for a different project), you probably don't need to do it
again.
## Code reviews
All submissions, including submissions by project members, require review. We
use GitHub pull requests for this purpose. Consult
[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more
information on using pull requests.

202
LICENSE Normal file
View file

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

54
README.md Normal file
View file

@ -0,0 +1,54 @@
# ForwardProxy plugin for Caddy webserver
This plugin enables Caddy webserver to act as a ForwardProxy for http/2.0 and http/1.1 requests
(http/1.0 might work, but is untested).
## ForwardProxy Caddyfile directives
To simply enable forward proxy without authentication just include the ```forwardproxy``` directive in your Caddyfile.
To do more advanced things, you may use expanded syntax:
```
forwardproxy {
basicauth caddyuser1 0NtCL2JPJBgPPMmlPcJ
basicauth caddyuser2 秘密
ports 80 443
hide_ip
experimental_probe_resist secretlink-7qS4+3dqm.localhost
response_timeout 30
dial_timeout 30
}
```
Warning: all directives are subject to changes!
* basicauth user password
Sets basic HTTP auth credentials. This directive may be repeated multiple times.
Default: no auth required.
* ports integer integer...
Whitelists ports forwardproxy will HTTP CONNECT to.
Default: no restrictions.
* hide_ip
If set, forwardproxy will not add user's IP to "Forwarded:" header.
Default: no hiding, "_Forwarded: for="useraddress"_" will be sent out.
* experimental_probe_resist secretlink.tld
EXPERIMENTAL, HERE BE DRAGONS.
Attempts to hide the fact that the site is a forwardproxy.
Proxy will no longer respond with _"407 Proxy Authentication Required"_ if credentials are incorrect or absent,
and will attempt to mimic generic forwardproxy-less Caddy server in other regards.
Not all clients(browsers) are able to be configured to send credentials right away,
and only provide credentials after receiving 407.
To work around this, we will use a secret link - the only link that will trigger 407 response,
prompting browsers to request credentials from users and cache them for the rest of the session.
It is possible to use any top level domain, but for secrecy reasons it is highly recommended to use .localhost.
Probing resistance works(and makes sense) only if basicauth is set up.
Default: no probing resistance.
* response_timeout integer
Sets timeout (in seconds) for HTTP requests made by proxy on behalf of users (does not affect CONNECT requests)
Default: no timeout(other timeouts will eventually close the connection).
* dial_timeout integer
Sets timeout (in seconds) for establishing TCP connection to target website. Affects all requests.
Default: 20 seconds.
### License
Licensed under the [Apache License](LICENSE)
Initial version of this plugin was developed by Google.
Disclaimer: This is not an official Google product.

277
common_test.go Normal file
View file

@ -0,0 +1,277 @@
package forwardproxy
import (
"crypto/tls"
"encoding/hex"
"fmt"
"github.com/mholt/caddy"
"io"
"io/ioutil"
"net/http"
"net/http/httputil"
"os"
"strings"
"testing"
"time"
)
var credentialsEmpty = ""
var credentialsCorrect = "Basic dGVzdDpwYXNz" // test:pass
var credentialsWrong = []string{
"",
"\"\"",
"Basic dzp3",
"Basic \"\"",
"Foo bar",
"Tssssssss",
"Basic dpz3 asp",
}
/*
Test naming: Test{httpVer}Proxy{Method}{Auth}{Credentials}{httpVer}
GET/CONNECT -- get gets, connect connects and gets
Auth/NoAuth
Empty/Correct/Wrong -- tries different credentials
*/
var testResources = []string{"", "/pic.png"}
var testHttpVersions = []string{"HTTP/2.0", "HTTP/1.1"}
type caddyTestServer struct {
*caddy.Instance
addr string // could be http or https
HTTPRedirectPort string // used in probe-resist tests to simulate default Caddy's http->https redirect
root string // expected to have index.html and pic.png
directives []string
proxyEnabled bool
proxyDirectives []string
contents map[string][]byte
}
var (
caddyForwardProxy caddyTestServer
caddyForwardProxyAuth caddyTestServer // requires auth
caddyForwardProxyProbeResist caddyTestServer // requires auth, and has probing resistance on
caddyDummyProbeResist caddyTestServer // same as caddyForwardProxyProbeResist, but w/o forwardproxy
caddyTestTarget caddyTestServer
)
func (c *caddyTestServer) marshal() []byte {
mainBlock := []string{c.addr + " {",
"root " + c.root}
mainBlock = append(mainBlock, c.directives...)
if c.proxyEnabled {
if len(c.proxyDirectives) == 0 {
mainBlock = append(mainBlock, "forwardproxy")
} else {
forwardProxyBlock := []string{"forwardproxy {"}
forwardProxyBlock = append(forwardProxyBlock, strings.Join(c.proxyDirectives, "\n"))
forwardProxyBlock = append(forwardProxyBlock, "}")
mainBlock = append(mainBlock, strings.Join(forwardProxyBlock, "\n"))
}
}
mainBlock = append(mainBlock, "}")
if len(c.HTTPRedirectPort) > 0 {
// TODO: this is not good enough, since `func redirPlaintextHost(cfg *SiteConfig) *SiteConfig`
// https://github.com/mholt/caddy/blob/master/caddyhttp/httpserver/https.go#L142 can change in future
// and we won't know.
redirectBlock := []string{"http://*:" + c.HTTPRedirectPort + " {",
"redir https://" + c.addr + "{uri}",
"header / Connection close",
"}"}
mainBlock = append(mainBlock, redirectBlock...)
}
// fmt.Println(strings.Join(mainBlock, "\n"))
return []byte(strings.Join(mainBlock, "\n"))
}
func (c *caddyTestServer) StartTestServer() {
var err error
c.Instance, err = caddy.Start(caddy.CaddyfileInput{Contents: c.marshal(), ServerTypeName: "http"})
if err != nil {
panic(err)
}
if c.contents == nil {
c.contents = make(map[string][]byte)
}
index, err := ioutil.ReadFile(c.root + "/index.html")
if err != nil {
panic(err)
}
c.contents[""] = index
c.contents["/"] = index
c.contents["/index.html"] = index
c.contents["/pic.png"], err = ioutil.ReadFile(c.root + "/pic.png")
if err != nil {
panic(err)
}
}
func TestMain(m *testing.M) {
caddyForwardProxy = caddyTestServer{addr: "127.0.0.1:1984", root: "./test/forwardproxy",
directives: []string{"tls self_signed"},
proxyEnabled: true}
caddyForwardProxy.StartTestServer()
caddyForwardProxyAuth = caddyTestServer{addr: "127.0.0.1:4891", root: "./test/forwardproxy",
directives: []string{"tls self_signed"},
proxyEnabled: true, proxyDirectives: []string{"basicauth test pass"}}
caddyForwardProxyAuth.StartTestServer()
caddyForwardProxyProbeResist = caddyTestServer{addr: "127.0.0.1:8888", root: "./test/forwardproxy",
directives: []string{"tls self_signed"}, HTTPRedirectPort: "8880",
proxyEnabled: true, proxyDirectives: []string{"basicauth test pass", "experimental_probe_resist test.localhost"}}
caddyForwardProxyProbeResist.StartTestServer()
caddyDummyProbeResist = caddyTestServer{addr: "127.0.0.1:9999", root: "./test/forwardproxy",
directives: []string{"tls self_signed"}, HTTPRedirectPort: "9980",
proxyEnabled: false}
caddyDummyProbeResist.StartTestServer()
// 127.0.0.1 and localhost are both used to avoid Caddy matching and routing proxy requests internally
caddyTestTarget = caddyTestServer{addr: "localhost:6451", root: "./test/index",
directives: []string{},
proxyEnabled: false}
caddyTestTarget.StartTestServer()
retCode := m.Run()
caddyForwardProxy.Stop()
caddyForwardProxyAuth.Stop()
caddyForwardProxyProbeResist.Stop()
caddyDummyProbeResist.Stop()
caddyTestTarget.Stop()
os.Exit(retCode)
}
// This is a sanity check confirming that target servers actually directly serve what they are expected to.
// (And that they don't serve what they should not)
func TestTheTest(t *testing.T) {
tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
ResponseHeaderTimeout: 2 * time.Second,
}
client := &http.Client{Transport: tr, Timeout: 2 * time.Second}
// Request index
resp, err := client.Get("http://" + caddyTestTarget.addr)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(resp, caddyTestTarget.contents[""]); err != nil {
t.Fatal(err)
}
// Request pic
resp, err = client.Get("http://" + caddyTestTarget.addr + "/pic.png")
if err != nil {
t.Fatal(err)
} else if err = responseExpected(resp, caddyTestTarget.contents["/pic.png"]); err != nil {
t.Fatal(err)
}
// Request pic, but expect index. Should fail
resp, err = client.Get("http://" + caddyTestTarget.addr + "/pic.png")
if err != nil {
t.Fatal(err)
} else if err = responseExpected(resp, caddyTestTarget.contents[""]); err == nil {
t.Fatal(err)
}
// Request index, but expect pic. Should fail
resp, err = client.Get("http://" + caddyTestTarget.addr)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(resp, caddyTestTarget.contents["/pic.png"]); err == nil {
t.Fatal(err)
}
// Request non-existing resource
resp, err = client.Get("http://" + caddyTestTarget.addr + "/idontexist")
if err != nil {
t.Fatal(err)
} else if resp.StatusCode != http.StatusNotFound {
t.Fatalf("Expected: 404 StatusNotFound, got %s. Response: %#v\n", resp.StatusCode, resp)
}
}
func TestIsSubdomain(t *testing.T) {
testSubDomain := func(s, domain string, expectedResult bool) {
result := isSubdomain(s, domain)
if result != expectedResult {
t.Fatalf("Expected: isSubdomain(%s, %s) is %b, Got: %b", s, domain, expectedResult, result)
}
}
testSubDomain("hoooli.abc", "hooya.ya", false)
testSubDomain("", "hooya.ya", false)
testSubDomain("hoooli.abc", "", false)
testSubDomain("hoooli.abc", "hiddenlink.localhost", false)
testSubDomain("www.hoooli.abc", "hoooli.abc", true)
testSubDomain("hoooli.abc", "hoooli.abc", true)
testSubDomain(".hoooli.abc", "hoooli.abc", true)
testSubDomain("sup.hoooli.abc", "hoooli.abc", true)
testSubDomain("qwe.qwe.qwe.hoooli.abc", "hoooli.abc", true)
}
func debugIoCopy(dst io.Writer, src io.Reader, prefix string) (written int64, err error) {
buf := make([]byte, 32*1024)
flusher, ok := dst.(http.Flusher)
for {
nr, er := src.Read(buf)
fmt.Printf("[%s] Read err %#v\n%s", prefix, er, hex.Dump(buf[0:nr]))
if nr > 0 {
nw, ew := dst.Write(buf[0:nr])
if ok {
flusher.Flush()
}
fmt.Printf("[%s] Wrote %v %v\n", prefix, nw, ew)
if nw > 0 {
written += int64(nw)
}
if ew != nil {
err = ew
break
}
if nr != nw {
err = io.ErrShortWrite
break
}
}
if er != nil {
if er != io.EOF {
err = er
}
break
}
}
fmt.Printf("[%s] Returning with %#v %#v\n", prefix, written, err)
return
}
func httpdump(r interface{}) string {
switch v := r.(type) {
case *http.Request:
if v == nil {
return "httpdump: nil"
}
b, err := httputil.DumpRequest(v, false)
if err != nil {
return err.Error()
} else {
return string(b)
}
case *http.Response:
if v == nil {
return "httpdump: nil"
}
b, err := httputil.DumpResponse(v, false)
if err != nil {
return err.Error()
} else {
return string(b)
}
default:
return "httpdump: wrong type"
}
}

16
docker-build/Dockerfile Normal file
View file

@ -0,0 +1,16 @@
FROM alpine:3.6
LABEL description="Docker image for caddy+forwardproxy plugin."
LABEL maintainer="SergeyFrolov@colorado.edu"
RUN apk add --no-cache ca-certificates bash curl
RUN curl --fail https://getcaddy.com | bash -s http.forwardproxy
COPY gen_caddyfile_and_start.sh /bin/
VOLUME /root/.caddy
EXPOSE 80 443 2015
ENTRYPOINT /bin/gen_caddyfile_and_start.sh

7
docker-build/README.md Normal file
View file

@ -0,0 +1,7 @@
# caddy-forwardproxy
A docker image for Caddy web server + forwardproxy plugin.
Allows to easily set up private web server with proxying.
### Build
```docker build -t caddy-forwardproxy .```
### Usage
Please find latest usage instructions in [run.sh](./run.sh).

32
docker-build/gen_caddyfile_and_start.sh Executable file
View file

@ -0,0 +1,32 @@
#!/usr/bin/env bash
CADDYFILE="${CADDYFILE:-/etc/caddy/Caddyfile}"
ROOTDIR="${ROOTDIR:-/srv/index}"
SITE_ADDRESS="${SITE_ADDRESS:-localhost}"
generate_caddyfile() {
mkdir -p "$(dirname "${CADDYFILE}")"
echo "${SITE_ADDRESS} {" > ${CADDYFILE}
echo " root $ROOTDIR" >> ${CADDYFILE}
echo " forwardproxy {" >> ${CADDYFILE}
if [[ ! -z ${PROXY_USERNAME} ]]; then
echo " basicauth ${PROXY_USERNAME} ${PROXY_PASSWORD}" >> ${CADDYFILE}
fi
if [[ "${PROBE_RESISTANT}" = true ]]; then
echo " experimental_probe_resist ${SECRET_LINK}" >> ${CADDYFILE}
fi
echo " }" >> ${CADDYFILE}
echo "}" >> ${CADDYFILE}
}
if [ -f "${CADDYFILE}" ]; then
echo "Using provided Caddyfile"
else
echo "Caddyfile is not provided: generating new one"
generate_caddyfile
fi
caddy ${CADDY_OPTS} -conf ${CADDYFILE}

39
docker-build/run.sh Executable file
View file

@ -0,0 +1,39 @@
#!/usr/bin/env bash
print_help() {
cat <<EOF
All arguments to this script are passed to docker.
One can configure docker image by setting variables and mounting folders.
To set address of served website:
-e SITE_ADDRESS=(string)
To set up credentials for your forwardproxy:
-e PROXY_USERNAME=(string) -e PROXY_PASSWORD=(string)
To enable probing resistance, and specify (optional) secret link:
-e PROBE_RESISTANT=true -e SECRET_LINK=(string)
To manually provide Caddyfile(and ignore all of above):
-v (path to Caddyfile):/etc/caddy/Caddyfile
To set served files:
-v (path to files):/srv/index
To persistently save certificates and avoid LE issuance limit:
-v (path to some storage folder):/root/.caddy
One can pass options to caddy using CADDY_OPTS e.g.:
-e CADDY_OPTS="-ca https://acme-staging.api.letsencrypt.org/directory"
One can also directly pass here other useful docker commands, e.g.:
--restart always
EOF
}
if [[ $1 == "help" || $1 == "--help" || $1 == "-h" ]]; then
print_help
exit 0
fi
docker run -p 2015:2015 -p 443:443 -p 80:80 "$@" caddy-forwardproxy

389
forwardproxy.go Normal file
View file

@ -0,0 +1,389 @@
// Copyright 2017 Google Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Caching is purposefully ignored. Pipelining is expected to work, but doesn't have to. Might be (ab)used to get
// into internal networks.
package forwardproxy
import (
"crypto/subtle"
"errors"
"fmt"
"io"
"net"
"net/http"
"strconv"
"strings"
"sync"
"time"
"github.com/mholt/caddy/caddyhttp/httpserver"
)
type ForwardProxy struct {
httpTransport http.Transport
Next httpserver.Handler
authRequired bool
authCredentials [][]byte // slice with base64-encoded credentials
hideIP bool
whitelistedPorts []int
probeResistDomain string
probeResistEnabled bool
dialTimeout time.Duration // for initial tcp connection
hostname string // do not intercept requests to the hostname (except for hidden link)
}
var bufferPool sync.Pool
// TODO?: getStatusCode(err) that casts to http.Error, net Error, etc. and returns correct http status code
func (fp ForwardProxy) connectPortIsAllowed(port string) bool {
portInt, err := strconv.Atoi(port)
if err != nil {
return false
}
if portInt <= 0 || portInt > 65535 {
return false
}
if len(fp.whitelistedPorts) == 0 {
return true
}
isAllowed := false
for _, p := range fp.whitelistedPorts {
if p == portInt {
isAllowed = true
break
}
}
return isAllowed
}
// Copies data r1->w1 and r2->w2, flushes as needed, and returns when both streams are done.
func dualStream(w1 io.Writer, r1 io.Reader, w2 io.Writer, r2 io.Reader) error {
errChan := make(chan error)
stream := func(w io.Writer, r io.Reader) {
buf := bufferPool.Get().([]byte)
buf = buf[0:cap(buf)]
_, _err := flushingIoCopy(w, r, buf)
errChan <- _err
}
go stream(w1, r1)
go stream(w2, r2)
err1 := <-errChan
err2 := <-errChan
if err1 != nil {
return err1
}
return err2
}
// Hijacks the connection from ResponseWriter, writes the response and proxies data between targetConn
// and hijacked connection.
func serveHijack(w http.ResponseWriter, targetConn net.Conn) (int, error) {
hijacker, ok := w.(http.Hijacker)
if !ok {
return http.StatusInternalServerError, errors.New("ResponseWriter does not implement Hijacker")
}
clientConn, bufReader, err := hijacker.Hijack()
if err != nil {
return http.StatusInternalServerError, errors.New("failed to hijack: " + err.Error())
}
defer clientConn.Close()
// bufReader may contain unprocessed buffered data from the client.
if bufReader != nil {
// snippet borrowed from `proxy` plugin
if n := bufReader.Reader.Buffered(); n > 0 {
rbuf, err := bufReader.Reader.Peek(n)
if err != nil {
return http.StatusBadGateway, err
}
targetConn.Write(rbuf)
}
}
// Since we hijacked the connection, we lost the ability to write and flush headers via w.
// Let's handcraft the response and send it manually.
res := &http.Response{StatusCode: http.StatusOK,
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: make(http.Header),
}
res.Header.Set("Server", "Caddy")
err = res.Write(clientConn)
if err != nil {
return http.StatusInternalServerError, errors.New("failed to send response to client: " + err.Error())
}
return 0, dualStream(targetConn, clientConn, clientConn, targetConn)
}
// Returns nil error on successful credentials check.
func (fp *ForwardProxy) checkCredentials(r *http.Request) error {
pa := strings.Split(r.Header.Get("Proxy-Authorization"), " ")
if len(pa) != 2 {
return errors.New("Proxy-Authorization is required! Expected format: <type> <credentials>")
}
if strings.ToLower(pa[0]) != "basic" {
return errors.New("Auth type is not supported")
}
for _, creds := range fp.authCredentials {
if subtle.ConstantTimeCompare(creds, []byte(pa[1])) == 1 {
// Please do not consider this to be timing-attack-safe code. Simple equality is almost
// mindlessly substituted with constant time algo and there ARE known issues with this code,
// e.g. size of smallest credentials is guessable. TODO: protect from all the attacks! Hash?
return nil
}
}
return errors.New("Invalid credentials")
}
// returns true if `s` is `domain` or subdomain of `domain`. Inputs are expected to be sanitized.
func isSubdomain(s, domain string) bool {
if s == domain {
return true
}
if strings.HasSuffix(s, "."+domain) {
return true
}
return false
}
// borrowed from `proxy` plugin
func stripPort(address string) string {
// Keep in mind that the address might be a IPv6 address
// and thus contain a colon, but not have a port.
portIdx := strings.LastIndex(address, ":")
ipv6Idx := strings.LastIndex(address, "]")
if portIdx > ipv6Idx {
address = address[:portIdx]
}
return address
}
func serveHiddenPage(w http.ResponseWriter, authErr error) (int, error) {
const hiddenPage = `<html>
<head>
<title>Hidden Proxy Page</title>
</head>
<body>
<h1>Hidden Proxy Page!</h1>
%s<br/>
</body>
</html>`
const AuthFail = "Please authenticate yourself to the proxy."
const AuthOk = "Congratulations, you are succussfully authenticated to the proxy! Go browse all the things!"
if authErr != nil {
w.Header().Set("Proxy-Authenticate", "Basic")
w.WriteHeader(http.StatusProxyAuthRequired)
w.Write([]byte(fmt.Sprintf(hiddenPage, AuthFail)))
return 0, authErr
}
w.Write([]byte(fmt.Sprintf(hiddenPage, AuthOk)))
return 0, nil
}
func (fp *ForwardProxy) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
var authErr error
if fp.authRequired {
authErr = fp.checkCredentials(r)
}
if fp.probeResistEnabled && len(fp.probeResistDomain) > 0 && stripPort(r.Host) == fp.probeResistDomain {
return serveHiddenPage(w, authErr)
}
if isSubdomain(stripPort(r.Host), fp.hostname) && (r.Method != http.MethodConnect || authErr != nil) {
// Always pass non-CONNECT requests to hostname
// Pass CONNECT requests only if probe resistance is enabled and not authenticated
return fp.Next.ServeHTTP(w, r)
}
if authErr != nil {
if fp.probeResistEnabled {
// probe resistance is requested and requested URI does not match secret domain
httpserver.WriteSiteNotFound(w, r)
return 0, authErr // current Caddy behavior without forwardproxy
} else {
w.Header().Set("Proxy-Authenticate", "Basic")
return http.StatusProxyAuthRequired, authErr
}
}
if r.ProtoMajor != 1 && r.ProtoMajor != 2 {
return http.StatusHTTPVersionNotSupported, errors.New("Unsupported HTTP major version: " + strconv.Itoa(r.ProtoMajor))
}
if r.Method == http.MethodConnect {
if r.ProtoMajor == 2 {
if len(r.URL.Scheme) > 0 || len(r.URL.Path) > 0 {
return http.StatusBadRequest, errors.New("CONNECT request has :scheme or/and :path pseudo-header fields")
}
}
if !fp.connectPortIsAllowed(r.URL.Port()) {
return http.StatusForbidden, errors.New("CONNECT port not allowed for " + r.URL.String())
}
targetConn, err := net.DialTimeout("tcp", r.URL.Hostname()+":"+r.URL.Port(), fp.dialTimeout)
if err != nil {
return http.StatusBadGateway, errors.New(fmt.Sprintf("Dial %s failed: %v", r.URL.String(), err))
}
defer targetConn.Close()
switch r.ProtoMajor {
case 1: // http1: hijack the whole flow
return serveHijack(w, targetConn)
case 2: // http2: keep reading from "request" and writing into same response
defer r.Body.Close()
wFlusher, ok := w.(http.Flusher)
if !ok {
return http.StatusInternalServerError, errors.New("ResponseWriter doesn't implement Flusher()")
}
w.WriteHeader(http.StatusOK)
wFlusher.Flush()
return 0, dualStream(targetConn, r.Body, w, targetConn)
default:
panic("There was a check for http version, yet it's incorrect")
}
} else {
outReq, err := fp.generateForwardRequest(r)
if err != nil {
return http.StatusBadRequest, err
}
response, err := fp.httpTransport.RoundTrip(outReq)
if err != nil {
if response != nil {
if response.StatusCode != 0 {
return response.StatusCode, errors.New("failed to do RoundTrip(): " + err.Error())
}
}
return http.StatusBadGateway, errors.New("failed to do RoundTrip(): " + err.Error())
}
return 0, forwardResponse(w, response)
}
}
// Removes hop-by-hop headers, and writes response into ResponseWriter.
func forwardResponse(w http.ResponseWriter, response *http.Response) error {
w.Header().Del("Server") // remove Server: Caddy, append via instead
w.Header().Add("Via", strconv.Itoa(response.ProtoMajor)+"."+strconv.Itoa(response.ProtoMinor)+" caddy")
for header, values := range response.Header {
for _, val := range values {
w.Header().Add(header, val)
}
}
removeHopByHop(w.Header())
w.WriteHeader(response.StatusCode)
buf := bufferPool.Get().([]byte)
buf = buf[0:cap(buf)]
_, err := io.CopyBuffer(w, response.Body, buf)
response.Body.Close()
return err
}
// Based on http Request from client, generates new request to be forwarded to target server.
// Some fields are shallow-copied, thus genOutReq will mutate original request.
// If error is not nil - http.StatusBadRequest is to be sent to client.
func (fp *ForwardProxy) generateForwardRequest(inReq *http.Request) (*http.Request, error) {
// Scheme has to be appended to avoid `unsupported protocol scheme ""` error.
// `http://` is used, since this initial request itself is always HTTP, regardless of what client and server
// may speak afterwards.
if len(inReq.RequestURI) == 0 {
return nil, errors.New("malformed request: empty URI")
}
strUrl := inReq.RequestURI
if strUrl[0] == '/' {
strUrl = inReq.Host + strUrl
}
if !strings.Contains(strUrl, "://") {
strUrl = "http://" + strUrl
}
outReq, err := http.NewRequest(inReq.Method, strUrl, inReq.Body)
if err != nil {
return outReq, errors.New("failed to create NewRequest: " + err.Error())
}
for key, values := range inReq.Header {
for _, value := range values {
outReq.Header.Add(key, value)
}
}
removeHopByHop(outReq.Header)
if !fp.hideIP {
outReq.Header.Add("Forwarded", "for=\""+inReq.RemoteAddr+"\"")
}
// https://tools.ietf.org/html/rfc7230#section-5.7.1
outReq.Header.Add("Via", strconv.Itoa(inReq.ProtoMajor)+"."+strconv.Itoa(inReq.ProtoMinor)+" caddy")
return outReq, nil
}
var hopByHopHeaders = []string{
"Keep-Alive",
"Proxy-Authenticate",
"Proxy-Authorization",
"Upgrade",
"Connection",
"Proxy-Connection",
"Te",
"Trailer",
"Transfer-Encoding",
}
func removeHopByHop(header http.Header) {
connectionHeaders := header.Get("Connection")
for _, h := range strings.Split(connectionHeaders, ",") {
header.Del(strings.TrimSpace(h))
}
for _, h := range hopByHopHeaders {
header.Del(h)
}
}
// flushingIoCopy is analogous to buffering io.Copy(), but also attempts to flush on each iteration.
// If dst does not implement http.Flusher(e.g. net.TCPConn), it will do a simple io.CopyBuffer().
// Reasoning: http2ResponseWriter will not flush on its own, so we have to do it manually.
func flushingIoCopy(dst io.Writer, src io.Reader, buf []byte) (written int64, err error) {
flusher, ok := dst.(http.Flusher)
if !ok {
return io.CopyBuffer(dst, src, buf)
}
for {
nr, er := src.Read(buf)
if nr > 0 {
nw, ew := dst.Write(buf[0:nr])
flusher.Flush()
if nw > 0 {
written += int64(nw)
}
if ew != nil {
err = ew
break
}
if nr != nw {
err = io.ErrShortWrite
break
}
}
if er != nil {
if er != io.EOF {
err = er
}
break
}
}
return
}

321
forwardproxy_test.go Normal file
View file

@ -0,0 +1,321 @@
// Copyright 2017 Google Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package forwardproxy
import (
"bufio"
"crypto/tls"
"errors"
"fmt"
_ "github.com/mholt/caddy/caddyhttp/header"
_ "github.com/mholt/caddy/caddyhttp/httpserver"
_ "github.com/mholt/caddy/caddyhttp/redirect"
_ "github.com/mholt/caddy/caddyhttp/root"
"io"
"net"
"net/http"
"net/url"
"testing"
"time"
)
func dial(proxyAddr string, useTls bool) (net.Conn, error) {
if useTls {
return tls.Dial("tcp", proxyAddr, &tls.Config{InsecureSkipVerify: true})
} else {
return net.Dial("tcp", proxyAddr)
}
}
func getViaProxy(targetHost, resource, proxyAddr, httpTargetVer, proxyCredentials string, useTls bool) (*http.Response, error) {
proxyConn, err := dial(proxyAddr, useTls)
if err != nil {
return nil, err
}
return getResourceViaProxyConn(proxyConn, targetHost, resource, httpTargetVer, proxyCredentials)
}
// if connect is not successful - that response is returned, otherwise the requested resource
func connectAndGetViaProxy(targetHost, resource, proxyAddr, httpTargetVer, proxyCredentials, httpProxyVer string, useTls bool) (*http.Response, error) {
proxyConn, err := dial(proxyAddr, useTls)
if err != nil {
return nil, err
}
connectRequest := http.Request{Header: make(http.Header)}
if len(proxyCredentials) > 0 {
connectRequest.Header.Set("Proxy-Authorization", proxyCredentials)
}
connectRequest.Host = targetHost
connectRequest.URL, err = url.Parse("http://" + connectRequest.Host)
if err != nil {
return nil, err
}
connectRequest.RequestURI = connectRequest.Host
connectRequest.Method = "CONNECT"
switch httpProxyVer {
case "HTTP/2.0":
connectRequest.ProtoMajor = 2
connectRequest.ProtoMinor = 0
case "HTTP/1.1":
connectRequest.ProtoMajor = 1
connectRequest.ProtoMinor = 1
default:
panic("http2ProxyVer: " + httpProxyVer)
}
connectRequest.Proto = httpProxyVer
if len(proxyCredentials) > 0 {
connectRequest.Header.Set("Proxy-Authorization", proxyCredentials)
}
err = connectRequest.Write(proxyConn)
if err != nil {
return nil, err
}
connectResponse, err := http.ReadResponse(bufio.NewReader(proxyConn), &connectRequest)
if err != nil {
return connectResponse, err
}
if connectResponse.StatusCode != http.StatusOK {
return connectResponse, err
}
return getResourceViaProxyConn(proxyConn, targetHost, resource, httpTargetVer, proxyCredentials)
}
func getResourceViaProxyConn(proxyConn net.Conn, targetHost, resource, httpTargetVer, proxyCredentials string) (*http.Response, error) {
var err error
request := http.Request{Header: make(http.Header)}
if len(proxyCredentials) > 0 {
request.Header.Set("Proxy-Authorization", proxyCredentials)
}
request.Host = targetHost
request.URL, err = url.Parse("http://" + request.Host + resource)
if err != nil {
return nil, err
}
request.RequestURI = request.Host + resource
request.Method = "GET"
switch httpTargetVer {
case "HTTP/2.0":
request.ProtoMajor = 2
request.ProtoMinor = 0
case "HTTP/1.1":
request.ProtoMajor = 1
request.ProtoMinor = 1
default:
panic("http2TargetVer: " + httpTargetVer)
}
request.Proto = httpTargetVer
err = request.WriteProxy(proxyConn)
if err != nil {
return nil, err
}
return http.ReadResponse(bufio.NewReader(proxyConn), &request)
}
// If response is expected: returns nil.
func responseExpected(res *http.Response, expectedResponse []byte) error {
responseLen := len(expectedResponse) + 2 // 2 extra bytes is enough to detected that expectedResponse is longer
response := make([]byte, responseLen)
var nTotal int
for {
n, err := res.Body.Read(response[nTotal:])
nTotal += n
if err == io.EOF {
break
}
if err != nil {
panic(err)
}
if nTotal == responseLen {
return errors.New(fmt.Sprintf("nTotal == responseLen, but haven't seen io.EOF. Expected response: %s\nGot: %s\n",
expectedResponse, response))
}
}
response = response[:nTotal]
if len(expectedResponse) != len(response) {
return errors.New(fmt.Sprintf("Expected length: %d. Got thus far: %d. Expected response: %s\nGot: %s\n",
len(expectedResponse), len(response), expectedResponse, response))
}
for i := range response {
if response[i] != expectedResponse[i] {
return errors.New(fmt.Sprintf("Response mismatch at character #%d. Expected response: %s\nGot: %s\n",
i, expectedResponse, response))
}
}
return nil
}
func TestPassthrough(t *testing.T) {
tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
ResponseHeaderTimeout: 2 * time.Second,
}
client := &http.Client{Transport: tr, Timeout: 2 * time.Second}
resp, err := client.Get("https://" + caddyForwardProxy.addr)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(resp, caddyForwardProxy.contents[""]); err != nil {
t.Fatal(err)
}
resp, err = client.Get("https://" + caddyForwardProxy.addr + "/pic.png")
if err != nil {
t.Fatal(err)
} else if err = responseExpected(resp, caddyForwardProxy.contents["/pic.png"]); err != nil {
t.Fatal(err)
}
resp, err = client.Get("https://" + caddyForwardProxy.addr + "/idontexist")
if err != nil {
t.Fatal(err)
} else if resp.StatusCode != http.StatusNotFound {
t.Fatalf("Expected: 404 StatusNotFound, got %s. Response: %#v\n", resp.StatusCode, resp)
}
}
func TestGETNoAuth(t *testing.T) {
useTls := true
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := getViaProxy(caddyTestTarget.addr, resource, caddyForwardProxy.addr, httpTargetVer, credentialsEmpty, useTls)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(response, caddyTestTarget.contents[resource]); err != nil {
t.Fatal(err)
}
}
}
}
func TestGETAuthCorrect(t *testing.T) {
useTls := true
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := getViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyAuth.addr, httpTargetVer, credentialsCorrect, useTls)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(response, caddyTestTarget.contents[resource]); err != nil {
t.Fatal(err)
}
}
}
}
func TestGETAuthWrong(t *testing.T) {
useTls := true
for _, wrongCreds := range credentialsWrong {
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := getViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyAuth.addr, httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
if response.StatusCode != http.StatusProxyAuthRequired {
t.Fatalf("Expected response: 407 StatusProxyAuthRequired, Got: %d %s\n",
response.StatusCode, response.Status)
}
}
}
}
}
func TestProxySelfGet(t *testing.T) {
useTls := true
// GETNoAuth to self
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := getViaProxy(caddyForwardProxy.addr, resource, caddyForwardProxy.addr, httpTargetVer, credentialsEmpty, useTls)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(response, caddyForwardProxy.contents[resource]); err != nil {
t.Fatal(err)
}
}
}
// GETAuthCorrect to self
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := getViaProxy(caddyForwardProxyAuth.addr, resource, caddyForwardProxyAuth.addr, httpTargetVer, credentialsCorrect, useTls)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(response, caddyForwardProxyAuth.contents[resource]); err != nil {
t.Fatal(err)
}
}
}
}
// TODO: self TestProxySelfConnect.
// It requires tls-in-tls, which tests are not currently set up for.
// Low priority since this is a functionality issue, not security, and it would be easily caught in the wild.
func TestConnectNoAuth(t *testing.T) {
useTls := true
for _, httpProxyVer := range testHttpVersions {
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := connectAndGetViaProxy(caddyTestTarget.addr, resource, caddyForwardProxy.addr, httpTargetVer, credentialsEmpty, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(response, caddyTestTarget.contents[resource]); err != nil {
t.Fatal(err)
}
}
}
}
}
func TestConnectAuthCorrect(t *testing.T) {
useTls := true
for _, httpProxyVer := range testHttpVersions {
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := connectAndGetViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyAuth.addr, httpTargetVer, credentialsCorrect, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(response, caddyTestTarget.contents[resource]); err != nil {
t.Fatal(err)
}
}
}
}
}
func TestConnectAuthWrong(t *testing.T) {
useTls := true
for _, wrongCreds := range credentialsWrong {
for _, httpProxyVer := range testHttpVersions {
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := connectAndGetViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyAuth.addr, httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
if response.StatusCode != http.StatusProxyAuthRequired {
t.Fatalf("Expected response: 407 StatusProxyAuthRequired, Got: %d %s\n",
response.StatusCode, response.Status)
}
}
}
}
}
}

410
probe_resist_test.go Normal file
View file

@ -0,0 +1,410 @@
package forwardproxy
import (
"bytes"
"errors"
"fmt"
"io"
"net/http"
"strings"
"testing"
)
func TestGETAuthCorrectProbeResist(t *testing.T) {
useTls := true
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := getViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyProbeResist.addr, httpTargetVer, credentialsCorrect, useTls)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(response, caddyTestTarget.contents[resource]); err != nil {
t.Fatal(err)
}
}
}
}
func TestGETAuthWrongProbeResist(t *testing.T) {
useTls := true
for _, wrongCreds := range credentialsWrong {
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
responseProbeResist, err := getViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyProbeResist.addr, httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
// get response from reference server without forwardproxy and compare them
responseReference, err := getViaProxy(caddyTestTarget.addr, resource, caddyDummyProbeResist.addr, httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
// as a sanity check, get 407 from simple authenticated forwardproxy
responseForwardProxy, err := getViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyAuth.addr, httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
if responseProbeResist.StatusCode != http.StatusNotFound {
t.Fatalf("Expected response: 404 StatusNotFound, Got: %d %s\n",
responseProbeResist.StatusCode, responseProbeResist.Status)
}
if err = responsesAreEqual(responseProbeResist, responseReference); err != nil {
t.Fatal(err)
}
if err = responsesAreEqual(responseProbeResist, responseForwardProxy); err == nil {
t.Fatal("Responses from servers with and without forwardproxy are expected to be different.")
}
}
for _, resource := range testResources {
responseProbeResist, err := getViaProxy(caddyForwardProxyProbeResist.addr, resource, caddyForwardProxyProbeResist.addr, httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
// get response from reference server without forwardproxy and compare them
responseReference, err := getViaProxy(caddyDummyProbeResist.addr, resource, caddyDummyProbeResist.addr, httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
// as a sanity check, get 407 from simple authenticated forwardproxy
responseForwardProxy, err := getViaProxy(caddyForwardProxyAuth.addr, resource, caddyForwardProxyAuth.addr, httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
if responseProbeResist.StatusCode != http.StatusOK {
t.Fatalf("Expected response: 200 StatusOK, Got: %d %s\n",
responseProbeResist.StatusCode, responseProbeResist.Status)
}
if err = responsesAreEqual(responseProbeResist, responseReference); err != nil {
t.Fatal(err)
}
if err = responsesAreEqual(responseProbeResist, responseForwardProxy); err == nil {
t.Fatal("Responses from servers with and without forwardproxy are expected to be different.")
}
}
}
}
}
// test that responses on http redirect port are same
func TestGETAuthWrongProbeResistRedir(t *testing.T) {
useTls := false
for _, wrongCreds := range credentialsWrong {
for _, httpTargetVer := range testHttpVersions {
// request test target
for _, resource := range testResources {
responseProbeResist, err := getViaProxy(caddyTestTarget.addr, resource, stripPort(caddyForwardProxyProbeResist.addr)+":"+caddyForwardProxyProbeResist.HTTPRedirectPort,
httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
// get response from reference server without forwardproxy and compare them
responseReference, err := getViaProxy(caddyTestTarget.addr, resource, stripPort(caddyDummyProbeResist.addr)+":"+caddyDummyProbeResist.HTTPRedirectPort,
httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
if responseProbeResist.StatusCode != http.StatusMovedPermanently {
t.Fatalf("Expected response: 301 StatusMovedPermanently, Got: %d %s\n",
responseProbeResist.StatusCode, responseProbeResist.Status)
}
if err = responsesAreEqual(responseProbeResist, responseReference); err != nil {
t.Fatal(err)
}
}
// request self
for _, resource := range testResources {
responseProbeResist, err := getViaProxy(caddyForwardProxyProbeResist.addr, resource, stripPort(caddyForwardProxyProbeResist.addr)+":"+caddyForwardProxyProbeResist.HTTPRedirectPort,
httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
// get response from reference server without forwardproxy and compare them
responseReference, err := getViaProxy(caddyDummyProbeResist.addr, resource, stripPort(caddyDummyProbeResist.addr)+":"+caddyDummyProbeResist.HTTPRedirectPort,
httpTargetVer, wrongCreds, useTls)
if err != nil {
t.Fatal(err)
}
if responseProbeResist.StatusCode != http.StatusMovedPermanently {
t.Fatalf("Expected response: 301 StatusMovedPermanently, Got: %d %s\n",
responseProbeResist.StatusCode, responseProbeResist.Status)
}
if err = responsesAreEqual(responseProbeResist, responseReference); err != nil {
t.Fatal(err)
}
}
}
}
}
func TestConnectAuthCorrectProbeResist(t *testing.T) {
useTls := true
for _, httpProxyVer := range testHttpVersions {
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
response, err := connectAndGetViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyProbeResist.addr, httpTargetVer, credentialsCorrect, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
} else if err = responseExpected(response, caddyTestTarget.contents[resource]); err != nil {
t.Fatal(err)
}
}
}
}
}
func TestConnectAuthWrongProbeResist(t *testing.T) {
useTls := true
for _, wrongCreds := range credentialsWrong {
for _, httpProxyVer := range testHttpVersions {
for _, httpTargetVer := range testHttpVersions {
for _, resource := range testResources {
responseProbeResist, err := connectAndGetViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyProbeResist.addr, httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
// get response from reference server without forwardproxy and compare them
responseReference, err := connectAndGetViaProxy(caddyTestTarget.addr, resource, caddyDummyProbeResist.addr, httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
// as a sanity check, get 407 from simple authenticated forwardproxy
responseForwardProxy, err := connectAndGetViaProxy(caddyTestTarget.addr, resource, caddyForwardProxyAuth.addr, httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
if responseProbeResist.StatusCode != http.StatusNotFound {
t.Fatalf("Expected response: 404 StatusNotFound, Got: %d %s\n",
responseProbeResist.StatusCode, responseProbeResist.Status)
}
if err = responsesAreEqual(responseProbeResist, responseReference); err != nil {
t.Fatal(err)
}
if err = responsesAreEqual(responseProbeResist, responseForwardProxy); err == nil {
t.Fatal("Responses from servers with and without forwardproxy are expected to be different.")
}
}
// request self
for _, resource := range testResources {
responseProbeResist, err := connectAndGetViaProxy(caddyForwardProxyProbeResist.addr, resource, caddyForwardProxyProbeResist.addr, httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
// get response from reference server without forwardproxy and compare them
responseReference, err := connectAndGetViaProxy(caddyDummyProbeResist.addr, resource, caddyDummyProbeResist.addr, httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
// as a sanity check, get 407 from simple authenticated forwardproxy
responseForwardProxy, err := connectAndGetViaProxy(caddyForwardProxyAuth.addr, resource, caddyForwardProxyAuth.addr, httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
if responseProbeResist.StatusCode != http.StatusOK {
t.Fatalf("Expected response: 200 StatusOK, Got: %d %s\n",
responseProbeResist.StatusCode, responseProbeResist.Status)
}
if err = responsesAreEqual(responseProbeResist, responseReference); err != nil {
t.Fatal(err)
}
if err = responsesAreEqual(responseProbeResist, responseForwardProxy); err == nil {
t.Fatal("Responses from servers with and without forwardproxy are expected to be different.")
}
}
}
}
}
}
// test that responses on http redirect port are same
func TestConnectAuthWrongProbeResistRedir(t *testing.T) {
useTls := false
for _, wrongCreds := range credentialsWrong {
for _, httpProxyVer := range testHttpVersions {
for _, httpTargetVer := range testHttpVersions {
// request test target
for _, resource := range testResources {
responseProbeResist, err := connectAndGetViaProxy(caddyTestTarget.addr, resource, stripPort(caddyForwardProxyProbeResist.addr)+":"+caddyForwardProxyProbeResist.HTTPRedirectPort,
httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
// get response from reference server without forwardproxy and compare them
responseReference, err := connectAndGetViaProxy(caddyTestTarget.addr, resource, stripPort(caddyDummyProbeResist.addr)+":"+caddyDummyProbeResist.HTTPRedirectPort,
httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
if responseProbeResist.StatusCode != http.StatusMovedPermanently {
t.Fatalf("Expected response: 301 StatusMovedPermanently, Got: %d %s\n",
responseProbeResist.StatusCode, responseProbeResist.Status)
}
if err = responsesAreEqual(responseProbeResist, responseReference); err != nil {
t.Fatal(err)
}
}
// request self
for _, resource := range testResources {
responseProbeResist, err := connectAndGetViaProxy(caddyForwardProxyProbeResist.addr, resource, stripPort(caddyForwardProxyProbeResist.addr)+":"+caddyForwardProxyProbeResist.HTTPRedirectPort,
httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
// get response from reference server without forwardproxy and compare them
responseReference, err := connectAndGetViaProxy(caddyDummyProbeResist.addr, resource, stripPort(caddyDummyProbeResist.addr)+":"+caddyDummyProbeResist.HTTPRedirectPort,
httpTargetVer, wrongCreds, httpProxyVer, useTls)
if err != nil {
t.Fatal(err)
}
if responseProbeResist.StatusCode != http.StatusMovedPermanently {
t.Fatalf("Expected response: 301 StatusMovedPermanently, Got: %d %s\n",
responseProbeResist.StatusCode, responseProbeResist.Status)
}
if err = responsesAreEqual(responseProbeResist, responseReference); err != nil {
t.Fatal(err)
}
}
}
}
}
}
// returns nil if are equal
func responsesAreEqual(res1, res2 *http.Response) error {
if res1 == nil {
return errors.New("res1 is nil")
}
if res2 == nil {
return errors.New("res2 is nil")
}
if res1.Status != res2.Status {
return errors.New("Status is different")
}
if res1.StatusCode != res2.StatusCode {
return errors.New("StatusCode is different")
}
if res1.ProtoMajor != res2.ProtoMajor {
return errors.New("ProtoMajor is different")
}
if res1.Close != res2.Close {
return errors.New("Close is different")
}
if res1.ProtoMinor != res2.ProtoMinor {
return errors.New("ProtoMinor is different")
}
if res1.ContentLength != res2.ContentLength {
return errors.New("ContentLength is different")
}
if res1.Uncompressed != res2.Uncompressed {
return errors.New("Uncompressed is different")
}
if res1.Proto != res2.Proto {
return errors.New("Proto is different")
}
if len(res1.TransferEncoding) != len(res2.TransferEncoding) {
return errors.New("TransferEncodings have different length")
}
// returns "" if equal
stringSlicesAreEqual := func(s1, s2 []string) string {
if s1 == nil && s2 == nil {
return ""
}
if s1 == nil {
return "s1 is nil, whereas s2 is not"
}
if s2 == nil {
return "s2 is nil, whereas s1 is not"
}
if len(s1) != len(s2) {
return fmt.Sprintf("different length: %d vs %d", len(s1), len(s2))
}
for i := range s1 {
if s1[i] != s2[i] {
return fmt.Sprintf("different string at position %d: %s vs %s", i, s1[i], s2[i])
}
}
return ""
}
errStr := stringSlicesAreEqual(res1.TransferEncoding, res2.TransferEncoding)
if errStr != "" {
return errors.New("TransferEncodings are different: " + errStr)
}
if len(res1.Header) != len(res2.Header) {
return errors.New("Headers have different length")
}
for k1, v1 := range res1.Header {
k1Lower := strings.ToLower(k1)
if k1Lower == "date" {
continue
}
v2, ok := res2.Header[k1]
if !ok {
return errors.New(fmt.Sprintf("Header \"%s: %s\" is absent in res2", k1, v1))
}
if k1Lower == "location" {
for i, h := range v2 {
v2[i] = removeAddressesStr(h)
}
for i, h := range v1 {
v1[i] = removeAddressesStr(h)
}
}
if errStr = stringSlicesAreEqual(v1, v2); errStr != "" {
return errors.New(fmt.Sprintf("Header \"%s\" is different: %s", k1, errStr))
}
}
// Compare bodies
buf1 := make([]byte, 2048)
buf2 := make([]byte, 2048)
var n1, n2 int
var err1, err2 error
makeBodyError := func(s string) error {
return errors.New(fmt.Sprintf("Bodies are different: %s. n1 = %d, n2 = %d. err1 = %v, err2 = %v. buf1 = %s, buf2 = %s",
s, n1, n2, err1, err2, buf1[:n1], buf2[:n2]))
}
for {
n1, err1 = res1.Body.Read(buf1[:])
n2, err2 = res2.Body.Read(buf2[:n1])
buf1 = removeAddressesByte(buf1[:n1])
buf2 = removeAddressesByte(buf2[:n1])
for i := range buf1 {
if buf1[i] != buf2[i] {
return makeBodyError(fmt.Sprintf("Mismatched character %d", i))
}
}
if err1 == io.EOF && err2 == io.EOF {
break
}
if err1 == io.EOF && err2 == nil {
_n, _ := res2.Body.Read(buf2[n1:])
n2 += _n
return makeBodyError("Body 2 is longer")
}
if err1 != nil || err2 != nil {
return makeBodyError("Unexpected Read errors")
}
}
return nil
}
// Responses from forwardproxy + proberesist and generic caddy can have different addresses present in headers.
// To avoid false positives - remove addresses before comparing.
func removeAddressesByte(b []byte) []byte {
b = bytes.Replace(b, []byte(caddyForwardProxyProbeResist.addr),
bytes.Repeat([]byte{'#'}, len(caddyForwardProxyProbeResist.addr)), -1)
b = bytes.Replace(b, []byte(caddyDummyProbeResist.addr),
bytes.Repeat([]byte{'#'}, len(caddyDummyProbeResist.addr)), -1)
return b
}
func removeAddressesStr(s string) string {
return string(removeAddressesByte([]byte(s)))
}

151
setup.go Normal file
View file

@ -0,0 +1,151 @@
// Copyright 2017 Google Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package forwardproxy
import (
"encoding/base64"
"errors"
"github.com/mholt/caddy"
"github.com/mholt/caddy/caddyhttp/httpserver"
"net"
"net/http"
"strconv"
"strings"
"sync"
"time"
)
func setup(c *caddy.Controller) error {
httpserver.GetConfig(c).FallbackSite = true
fp := &ForwardProxy{dialTimeout: time.Second * 20, hostname: httpserver.GetConfig(c).Host(),
httpTransport: *http.DefaultTransport.(*http.Transport)}
fp.httpTransport.DialTLS = func(network, addr string) (net.Conn, error) {
return nil, &http.ProtocolError{ErrorString: "Proxy does not fetch TLS resources, use CONNECT instead"}
}
c.Next() // skip the directive name
args := c.RemainingArgs()
if len(args) > 0 {
return c.ArgErr()
}
for c.NextBlock() {
subdirective := c.Val()
args := c.RemainingArgs()
switch subdirective {
case "basicauth":
if len(args) != 2 {
return c.ArgErr()
}
if len(args[0]) == 0 {
return errors.New("Parse error: empty usernames are not allowed")
}
// TODO: Evaluate policy of allowing empty passwords.
if strings.Contains(args[0], ":") {
return errors.New("Parse error: character ':' in usernames is not allowed")
}
if fp.authCredentials == nil {
fp.authCredentials = [][]byte{}
}
// base64-encode credentials
buf := make([]byte, base64.StdEncoding.EncodedLen(len(args[0])+1+len(args[1])))
base64.StdEncoding.Encode(buf, []byte(args[0]+":"+args[1]))
fp.authCredentials = append(fp.authCredentials, buf)
fp.authRequired = true
case "ports":
if len(args) == 0 {
return c.ArgErr()
}
if len(fp.whitelistedPorts) != 0 {
return errors.New("Parse error: ports subdirective specified twice")
}
fp.whitelistedPorts = make([]int, len(args))
for i, p := range args {
intPort, err := strconv.Atoi(p)
if intPort <= 0 || intPort > 65535 || err != nil {
return errors.New("Parse error: ports are expected to be space-separated" +
" and in 0-65535 range. Got: " + p)
}
fp.whitelistedPorts[i] = intPort
}
case "hide_ip":
if len(args) != 0 {
return c.ArgErr()
}
fp.hideIP = true
case "experimental_probe_resist":
if len(args) > 1 {
return c.ArgErr()
}
fp.probeResistEnabled = true
if len(args) == 1 {
fp.probeResistDomain = args[0]
}
case "response_timeout":
if len(args) != 1 {
return c.ArgErr()
}
timeout, err := strconv.Atoi(args[0])
if err != nil {
return c.ArgErr()
}
if timeout < 0 {
return errors.New("Parse error: response_timeout cannot be negative.")
}
fp.httpTransport.ResponseHeaderTimeout = time.Second * time.Duration(timeout)
case "dial_timeout":
if len(args) != 1 {
return c.ArgErr()
}
timeout, err := strconv.Atoi(args[0])
if err != nil {
return c.ArgErr()
}
if timeout < 0 {
return errors.New("Parse error: dial_timeout cannot be negative.")
}
fp.dialTimeout = time.Second * time.Duration(timeout)
default:
return c.ArgErr()
}
}
if fp.probeResistEnabled && !fp.authRequired {
return errors.New("Parse error: probing resistance requires authentication")
}
fp.httpTransport.DialContext = (&net.Dialer{
Timeout: fp.dialTimeout,
KeepAlive: 30 * time.Second,
DualStack: true,
}).DialContext
httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
fp.Next = next
return fp
})
makeBuffer := func() interface{} { return make([]byte, 0, 32*1024) }
bufferPool = sync.Pool{New: makeBuffer}
return nil
}
func init() {
caddy.RegisterPlugin("forwardproxy", caddy.Plugin{
ServerType: "http",
Action: setup,
})
}

105
setup_test.go Normal file
View file

@ -0,0 +1,105 @@
// Copyright 2017 Google Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package forwardproxy
import (
"github.com/mholt/caddy"
"testing"
)
func TestSetup(t *testing.T) {
c := caddy.NewTestController("http", "forwardproxy string")
err := setup(c)
if err == nil {
t.Fatal("Expected: failure. Got: success. Input: forwardproxy string")
}
testParsing := func(subdirectives []string, shouldSucceed bool) {
input := "forwardproxy"
if len(subdirectives) > 0 {
input += " {\n"
for _, s := range subdirectives {
input += s + "\n"
}
input += "}"
}
c := caddy.NewTestController("http", input)
err := setup(c)
if shouldSucceed && err != nil {
t.Fatalf("Expected: success. Got: %v. Input:\n%s\n", err, input)
}
if !shouldSucceed && err == nil {
t.Fatalf("Expected: failure. Got: success. Input:\n%s\n", input)
}
}
testParsing(nil, true)
testParsing([]string{}, true)
testParsing([]string{"qweqwe"}, false)
testParsing([]string{"0"}, false)
testParsing([]string{"basicauth john"}, false)
testParsing([]string{"basicauth john \"\""}, true)
testParsing([]string{"basicauth john", "basicauth john \"\""}, false)
testParsing([]string{"basicauth john doe"}, true)
testParsing([]string{"basicauth john doe foo"}, false)
testParsing([]string{"basicauth john doe foo bar"}, false)
testParsing([]string{"basicauth \"\" doe"}, false)
testParsing([]string{"basicauth \"\" \"\""}, false)
testParsing([]string{"basicauth 0"}, false)
testParsing([]string{"basicauth 0 0"}, true)
testParsing([]string{"basicauth 0 0 0"}, false)
testParsing([]string{"basicauth 秘密"}, false)
testParsing([]string{"basicauth 秘密 秘密"}, true)
testParsing([]string{"basicauth 秘密 秘密 秘密"}, false)
testParsing([]string{"basicauth cyrillic пароль"}, true)
testParsing([]string{"basicauth john \"\"", "basicauth john doe", "basicauth 0 0", "basicauth 秘密 秘密", "basicauth cyrillic пароль"}, true)
testParsing([]string{"ports"}, false)
testParsing([]string{"ports 0"}, false)
testParsing([]string{"ports 0 1"}, false)
testParsing([]string{"ports -1"}, false)
testParsing([]string{"ports hi!"}, false)
testParsing([]string{"ports 11, 122, 33"}, false)
testParsing([]string{"ports 11, 122, 33"}, false)
testParsing([]string{"ports 11111 99999"}, false)
testParsing([]string{"ports 11 12"}, true)
testParsing([]string{"ports 1"}, true)
testParsing([]string{"ports 1 11 111 332 324 6546 33333"}, true)
testParsing([]string{"ports 1 11 111 332 324 6546 33333", "ports 1 11 111 332 324 6546 33333"}, false)
testParsing([]string{"ports 1", "ports 2"}, false)
testParsing([]string{"hide_ip"}, true)
testParsing([]string{"hide_ip 0"}, false)
testParsing([]string{"hide_ip 0 1"}, false)
testParsing([]string{"experimental_probe_resist"}, false)
testParsing([]string{"experimental_probe_resist local.host"}, false)
testParsing([]string{"experimental_probe_resist local.host very.local.host"}, false)
testParsing([]string{"experimental_probe_resist", "basicauth john doe"}, true)
testParsing([]string{"experimental_probe_resist local.host", "basicauth john doe"}, true)
testParsing([]string{"experimental_probe_resist local.host very.local.host", "basicauth john doe"}, false)
testParsing([]string{"response_timeout"}, false)
testParsing([]string{"response_timeout -1"}, false)
testParsing([]string{"response_timeout 1 2"}, false)
testParsing([]string{"response_timeout seven"}, false)
testParsing([]string{"response_timeout 2"}, true)
testParsing([]string{"dial_timeout"}, false)
testParsing([]string{"dial_timeout -1"}, false)
testParsing([]string{"dial_timeout 1 2"}, false)
testParsing([]string{"dial_timeout seven"}, false)
testParsing([]string{"dial_timeout 2"}, true)
}

1
test/forwardproxy/index.html Normal file
View file

@ -0,0 +1 @@
I am ForwardProxy(don't tell anyone)

BIN
test/forwardproxy/pic.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 21 KiB

1
test/index/index.html Normal file
View file

@ -0,0 +1 @@
I am not a ForwardProxy, but I want to be when I grow up!

BIN
test/index/pic.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 26 KiB