From 2974814d7d1320923a0e98a011cf31d3349352ac Mon Sep 17 00:00:00 2001 From: hustf Date: Sat, 9 Jun 2018 14:40:17 +0200 Subject: [PATCH] Add serve methods, clean-up mod:: README.md Add 'Errors?', 'Switch to HTTP' mod:: benchmark/benchmark.jl Include benchmark_prepare.jl mod:: benchmark/benchmark_prepare.jl Avoid redefinitions mod:: benchmark/functions_open_browsers.jl Mutable default address mod:: examples/chat_explore.jl usews(ws) -> coroutine(ws) new: examples/minimal_server.jl Readme.md example new: examples/minimal_server_HTTP.jl Readme.md example for HTTP deleted: examples/repl-client.html Delete bad practice deleted: examples/server.jl Delete insecure practice: eval. mod:: src/HTTP.jl Add methods for serve, fewer arguments. mod:: test/client_server_test.jl Use fewer arguments mod:: test/error_test.jl Use new methods serve --- README.md | 53 ++++++++++++++++---------- benchmark/benchmark.jl | 10 +++++ benchmark/benchmark_prepare.jl | 19 +++++++--- benchmark/functions_open_browsers.jl | 25 +++++++++---- examples/chat_explore.jl | 14 ++----- examples/minimal_server.jl | 33 ++++++++++++++++ examples/minimal_server_HTTP.jl | 33 ++++++++++++++++ examples/repl-client.html | 56 ---------------------------- examples/server.jl | 37 ------------------ src/HTTP.jl | 12 ++++-- test/client_server_test.jl | 2 +- test/error_test.jl | 6 +-- 12 files changed, 157 insertions(+), 143 deletions(-) create mode 100644 examples/minimal_server.jl create mode 100644 examples/minimal_server_HTTP.jl delete mode 100644 examples/repl-client.html delete mode 100644 examples/server.jl diff --git a/README.md b/README.md index b3b792b..8d712a7 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ Call `run`, which is a wrapper for calling `listen`. See inline docs. ##### Using HTTP Call `WebSockets.serve`, which is a wrapper for `HTTP.listen`. See inline docs. -## What does `WebSockets.jl` enable? +## What does WebSockets.jl enable? - reading and writing between entities you can program or know about - low latency messaging @@ -39,28 +39,28 @@ Call `WebSockets.serve`, which is a wrapper for `HTTP.listen`. See inline docs. - heartbeating, relaying - build a network including browser clients - convenience functions for gatekeeping with a common interface for HttpServer and HTTP -- writing http handlers and websockets 'handlers' in the same process can be an advantage. Exchanging unique tokens via http(s) - before accepting websockets is recommended for improved security. +- writing http handlers and websocket coroutines ('handlers') in the same process can be an advantage. Exchanging unique tokens via http(s) + before accepting websockets is recommended for improved security -WebSockets are well suited for user interactions via a browser. By calling compiled Javascript functions in browsers and using parallel workers, -user interaction and graphics workload, even development time can be moved off Julia resources. +WebSockets are well suited for user interactions via a browser or [cross-platform applications](https://electronjs.org/). User interaction and graphics workload, even development time can be moved off Julia resources. Use websockets to pass arguments between compiled functions on both sides; don't evaluate received code! -The /logutils folder contains some logging functionality that is quite fast and can make working with multiple asyncronous tasks easier. This functionality may be moved out of WebSockets in the future, depending on how other logging capabilities develop. +The /logutils folder contains some specialized logging functionality that is quite fast and can make working with multiple asyncronous tasks easier. See /benchmark code for how to use. Logging may be moved out of WebSockets in the future, depending on how other logging capabilities develop. -You should also have a look at Julia packages [DandelionWebSockets](https://github.com/dandeliondeathray/DandelionWebSockets.jl) or the implementation currently part of HTTP.jl. +You should also have a look at alternative Julia packages: [DandelionWebSockets](https://github.com/dandeliondeathray/DandelionWebSockets.jl) or the implementation currently part of HTTP.jl. ## What are the main downsides to WebSockets (in Julia)? - Logging. We need customizable and very fast logging for building networked applications. -- Security. Julia's Http(s) servers are currently not working to our knowledge. -- Non-compliant proxies on the internet, company firewalls. Commercial applications often use competing technologies for this reason, according to some old articles at least. HTTP.jl lets you use such techniques. -- 'Warm-up', i.e. compilation when a method is first used. These are excluded from current benchmarks. +- Security. Julia's Http(s) servers are currently not fully working to our knowledge. +- Compression is not implemented. +- Possibly non-compliant proxies on the internet, company firewalls. +- 'Warm-up', i.e. compilation when a method is first used. Warm-up is excluded from current benchmarks. - Garbage collection, which increases message latency at semi-random intervals. See benchmark plots. - If a connection is closed improperly, the connection task will throw uncaught ECONNRESET and similar messages. - TCP quirks, including 'warm-up' time with low transmission speed after a pause. Heartbeats can alleviate. -- Neither HTTP.jl or HttpServer.jl are made just for connecting WebSockets. You may need strong points from both. -- The optional dependencies increases load time compared to fixed dependencies. -- Since 'read' is a blocking function, you can easily end up reading indefinetely from both sides. +- Neither HTTP.jl or HttpServer.jl are made just for connecting WebSockets. You may need strong points from both. +- The optional dependencies may increase load time compared to fixed dependencies. +- Since 'read' is a blocking function, you can easily end up reading indefinitely from both sides. ## Server side example @@ -129,7 +129,7 @@ You could replace 'using HttpServer' with 'using HTTP'. Also: WebSocketHandler -> WebSockets.WebsocketHandler -## Client side +## Client side example You need to use [HTTP.jl](https://github.com/JuliaWeb/HttpServer.jl). @@ -170,14 +170,29 @@ WebSockets.open("ws://127.0.0.1:$PORT") do ws end ``` -The output in a console session is barely readable, which is irritating. To build real-time applications, we need more code. +The output from the example in a console session is barely readable. Output from asyncronous tasks are intermixed. To build real-time applications, we need more code. See other examples in /test, /benchmark/ and /examples. Some logging utilties for a running relay server are available in /logutils. - - - - +## Errors after updating? + +The introduction of client side websockets to this package may require changes in your code: +- `using HttpServer` (or import) prior to `using WebSockets` (or import). +- The `WebSocket.id` field is no longer supported. You can generate unique counters by code similar to 'bencmark/functions_open_browsers.jl' COUNTBROWSER. +- You may want to modify error handling code. Examine WebSocketsClosedError.message. +- You may want to use `readguarded` and `writeguarded` to save on error handling code. + +## Switching from HttpServer to HTTP? +Some types and methods are not exported. See inline docs: +- `Server` -> `WebSockets.ServerWS` +- `WebSocketHandler` -> `WebSockets.WebsocketHandler` +- `run` -> `WebSockets.serve()` +- `Response` -> `HTTP.Response` +- `Request` -> `HTTP.Response` +- `HttpHandler`-> `HTTP.HandlerFunction` + + You may also want to consider using `target`, `orgin`and `subprotocol`, which + are compatible with both of the types above. ~~~~ diff --git a/benchmark/benchmark.jl b/benchmark/benchmark.jl index afc7b68..c53fdc7 100644 --- a/benchmark/benchmark.jl +++ b/benchmark/benchmark.jl @@ -81,3 +81,13 @@ Outlined benchmarks for developing an application using WebSockets (postponed): choice of message size and buffers for long-running calculations which connects to a server for distribution of results =# +if !isdefined(:SRCPATH) + const SRCPATH = Base.source_dir() == nothing ? Pkg.dir("WebSockets", "benchmark") :Base.source_dir() + const LOGGINGPATH = realpath(joinpath(SRCPATH, "../logutils/")) + SRCPATH ∉ LOAD_PATH && push!(LOAD_PATH, SRCPATH) + LOGGINGPATH ∉ LOAD_PATH && push!(LOAD_PATH, LOGGINGPATH) + include(joinpath(SRCPATH, "functions_open_browsers.jl")) +end +# Don't run this if the output files are recent? +include("benchmark_prepare.jl") +# include detailed benchmarks using the results from the above as nominal. \ No newline at end of file diff --git a/benchmark/benchmark_prepare.jl b/benchmark/benchmark_prepare.jl index 01dc129..cce77a3 100644 --- a/benchmark/benchmark_prepare.jl +++ b/benchmark/benchmark_prepare.jl @@ -2,18 +2,25 @@ # This file collects data and tests functions. # The intention is preparing input for a benchmark suite, -# but the output actually is sufficient for most purposes. +# but the output from this file actually is sufficient for some purposes. # # Both log file, results plots and tables are printed to the same file. # Viewing the plots in a text editor is probably possible, see UnicodePlots.jl. -# Running this file on a Windows laptop with all browsers takes 5-10 minutes +# Pull request welcome if someone can figure out how to do it. +# Running this file on a Windows laptop with all browsers takes 5-10 minutes. + + +if !isdefined(:SRCPATH) + const SRCPATH = Base.source_dir() == nothing ? Pkg.dir("WebSockets", "benchmark") :Base.source_dir() + const LOGGINGPATH = realpath(joinpath(SRCPATH, "../logutils/")) + SRCPATH ∉ LOAD_PATH && push!(LOAD_PATH, SRCPATH) + LOGGINGPATH ∉ LOAD_PATH && push!(LOAD_PATH, LOGGINGPATH) + include(joinpath(SRCPATH, "functions_open_browsers.jl")) +end + "A vector of message sizes" const VSIZE = reverse([i^3 * 1020 for i = 1:12]) -const SRCPATH = Base.source_dir() == nothing ? Pkg.dir("WebSockets", "benchmark") :Base.source_dir() -const LOGGINGPATH = realpath(joinpath(SRCPATH, "../logutils/")) -SRCPATH ∉ LOAD_PATH && push!(LOAD_PATH, SRCPATH) -LOGGINGPATH ∉ LOAD_PATH && push!(LOAD_PATH, LOGGINGPATH) include(joinpath(SRCPATH, "functions_open_browsers.jl")) include(joinpath(SRCPATH, "functions_benchmark.jl")) # diff --git a/benchmark/functions_open_browsers.jl b/benchmark/functions_open_browsers.jl index d22492d..a262352 100644 --- a/benchmark/functions_open_browsers.jl +++ b/benchmark/functions_open_browsers.jl @@ -1,4 +1,12 @@ -# Included in benchmark_2.jl +# Included in benchmark_prepare.jl and in browsertests.jl +# Refers logutils +if !isdefined(:SRCPATH) + const SRCPATH = Base.source_dir() == nothing ? Pkg.dir("WebSockets", "benchmark") :Base.source_dir() + const LOGGINGPATH = realpath(joinpath(SRCPATH, "../logutils/")) + SRCPATH ∉ LOAD_PATH && push!(LOAD_PATH, SRCPATH) + LOGGINGPATH ∉ LOAD_PATH && push!(LOAD_PATH, LOGGINGPATH) +end +using logutils_ws "A list of potentially available browsers, to be tried in succession if present" const BROWSERS = ["chrome", "firefox", "iexplore", "safari", "phantomjs"] @@ -7,8 +15,11 @@ mutable struct Countbrowser;value::Int;end (c::Countbrowser)() =COUNTBROWSER.value += 1 "For next value: COUNTBROWSER(). For current value: COUNTBROWSER.value" const COUNTBROWSER = Countbrowser(0) -const PORT = 8000 -const URL = "http://127.0.0.1:$PORT/bce.html" +const PORT = [8000] +const PAGE = ["bce.html"] +const URL = ["http://127.0.0.1:$(PORT[1])/$(PAGE[1])"] + + "Get application path for developer applications" function fwhich(s) @@ -151,11 +162,11 @@ end "Try to open one browser from BROWSERS. In some cases we expect an immediate indication -of failure, for example when the corresponding file +of failure, for example when the corresponding browser is not found on the system. In other cases, we will -just wait in vain for a request. In those cases, -call this function again. It remembers which browsers -were tried before. +just wait in vain. In those cases, +call this function again after a reasonable timeout. +The function remembers which browsers were tried before. " function open_a_browser() id = "open_next_browser" diff --git a/examples/chat_explore.jl b/examples/chat_explore.jl index ea91935..6483c94 100644 --- a/examples/chat_explore.jl +++ b/examples/chat_explore.jl @@ -42,7 +42,7 @@ particular websocket is open. The argument is an open websocket. Other instances of the function run in other tasks. The tasks are generated by either HTTP or HttpServer. """ -function usews(thisws) +function coroutine(thisws) global lastws = thisws push!(WEBSOCKETS, thisws => length(WEBSOCKETS) +1 ) t1 = now() + CLOSEAFTER @@ -115,7 +115,7 @@ end """ -`Server => gatekeeper(Request, WebSocket) => usews(WebSocket)` +`Server => gatekeeper(Request, WebSocket) => coroutine(WebSocket)` The gatekeeper makes it a little harder to connect with malicious code. It inspects the request that was upgraded @@ -126,7 +126,7 @@ function gatekeeper(req, ws) global lastws = ws orig = WebSockets.origin(req) if startswith(orig, "http://localhost") || startswith(orig, "http://127.0.0.1") - usews(ws) + coroutine(ws) else warn("Unauthorized websocket connection, $orig not approved by gatekeeper") end @@ -143,7 +143,7 @@ server_httpserver = Server(HttpHandler(req2resp), WebSocketHandler(gatekeeper)) server_HTTP = WebSockets.ServerWS(HTTP.HandlerFunction(req2resp), WebSockets.WebsocketHandler(gatekeeper)) # Start the HTTP server asyncronously, and stop it later -litas_HTTP = @schedule WebSockets.serve(server_HTTP, URL, HTTPPORT, false) +litas_HTTP = @schedule WebSockets.serve(server_HTTP, URL, HTTPPORT) @schedule begin println("HTTP server listening on $URL:$HTTPPORT for $CLOSEAFTER") sleep(CLOSEAFTER.value) @@ -160,10 +160,4 @@ litas_httpserver = @schedule run(server_httpserver, HTTPSERVERPORT) Base.throwto(litas_httpserver, InterruptException()) end -# Note that stopping the HttpServer in a while will send an error messages to the -# console. We could get rid of the messages by binding the task to a Channel. -# However, we can't get rid of ECONNRESET messages in that way. This is -# because the errors are triggered in tasks generated by litas_httpserver again, -# and those aren't channeled anywhere. - nothing \ No newline at end of file diff --git a/examples/minimal_server.jl b/examples/minimal_server.jl new file mode 100644 index 0000000..f19ff2e --- /dev/null +++ b/examples/minimal_server.jl @@ -0,0 +1,33 @@ +using HttpServer +using WebSockets + +function coroutine(ws) + while isopen(ws) + data, = readguarded(ws) + s = String(data) + if s == "" + break + end + println(s) + if s[1] == "P" + writeguarded(ws, "No, I'm not!") + else + writeguarded(ws, "Why?") + end + end +end + +function gatekeeper(req, ws) + if origin(req) == "http://127.0.0.1:8080" || origin(req) == "http://localhost:8080" + coroutine(ws) + else + println(origin(req)) + end +end + +handle(req, res) = Response(200) + +server = Server(HttpHandler(handle), + WebSocketHandler(gatekeeper)) + +@async run(server, 8080) diff --git a/examples/minimal_server_HTTP.jl b/examples/minimal_server_HTTP.jl new file mode 100644 index 0000000..2450f5b --- /dev/null +++ b/examples/minimal_server_HTTP.jl @@ -0,0 +1,33 @@ +using HTTP +using WebSockets + +function coroutine(ws) + while isopen(ws) + data, = readguarded(ws) + s = String(data) + if s == "" + break + end + println(s) + if s[1] == "P" + writeguarded(ws, "No, I'm not!") + else + writeguarded(ws, "Why?") + end + end +end + +function gatekeeper(req, ws) + if origin(req) == "http://127.0.0.1:8080" || origin(req) == "http://localhost:8080" + coroutine(ws) + else + println(origin(req)) + end +end + +handle(req, res) = HTTP.Response(200) + +server = WebSockets.ServerWS(HTTP.HandlerFunction(handle), + WebSockets.WebsocketHandler(gatekeeper)) + +@async WebSockets.serve(server, 8080) diff --git a/examples/repl-client.html b/examples/repl-client.html deleted file mode 100644 index 91d4373..0000000 --- a/examples/repl-client.html +++ /dev/null @@ -1,56 +0,0 @@ - - - - - - - Websockets client - - - -
-
-
- -
- - - - - diff --git a/examples/server.jl b/examples/server.jl deleted file mode 100644 index 78ac7da..0000000 --- a/examples/server.jl +++ /dev/null @@ -1,37 +0,0 @@ -using HttpServer -using WebSockets - -function decodeMessage( msg ) - String(copy(msg)) -end -function eval_or_describe_error(strmsg) - try - return eval(parse(strmsg, raise = false)) - catch err - iob = IOBuffer() - dump(iob, err) - return String(take!(iob)) - end -end - -wsh = WebSocketHandler() do req, client - while true - val = client |> read |> decodeMessage |> eval_or_describe_error - output = String(take!(Base.mystreamvar)) - val = val == nothing ? "
" : val - write(client,"$val
$output") - end -end - -onepage = readstring(Pkg.dir("WebSockets","examples","repl-client.html")) -httph = HttpHandler() do req::Request, res::Response - Response(onepage) -end - -server = Server(httph, wsh) -println("Repl server listening on 8080...") - -eval(Base,parse("mystreamvar = IOBuffer()")) -eval(Base,parse("STDOUT = mystreamvar")) - -run(server,8080) diff --git a/src/HTTP.jl b/src/HTTP.jl index 4e95a1d..5de35ef 100644 --- a/src/HTTP.jl +++ b/src/HTTP.jl @@ -225,7 +225,7 @@ end """ - WebSockets.ServerWS(::HTTP.HandlerFunction, ::WebSockets.WebsocketHandler(gatekeeper)) + WebSockets.ServerWS(::HTTP.HandlerFunction, ::WebSockets.WebsocketHandler) WebSockets.ServerWS is an argument type for WebSockets.serve. Instances include .in and .out channels, see WebSockets.serve. @@ -261,13 +261,15 @@ function ServerWS(handler::H, end """ - WebSockets.serve(server::ServerWS{T, H, W}, host, port, verbose) + WebSockets.serve(server::ServerWS, port) + WebSockets.serve(server::ServerWS, host, port) + WebSockets.serve(server::ServerWS, host, port, verbose) A wrapper for HTTP.listen. Puts any caught error and stacktrace on the server.out channel. To stop a running server, put HTTP.Servers.KILL on the .in channel. ```julia - @shedule WebSockets.serve(server, "127.0.0.1", 8080, false) + @shedule WebSockets.serve(server, "127.0.0.1", 8080) ``` After a suspected connection task failure: ```julia @@ -314,4 +316,6 @@ function serve(server::ServerWS{T, H, W}, host, port, verbose) where {T, H, W} end end return -end \ No newline at end of file +end +serve(server::WebSockets.ServerWS, host, port) = serve(server, host, port, false) +serve(server::WebSockets.ServerWS, port) = serve(server, "127.0.0.1", port, false) \ No newline at end of file diff --git a/test/client_server_test.jl b/test/client_server_test.jl index 1b17a76..51d1bef 100644 --- a/test/client_server_test.jl +++ b/test/client_server_test.jl @@ -40,7 +40,7 @@ server_WS = WebSockets.ServerWS( HTTP.HandlerFunction(req-> HTTP.Response(200)), WebSockets.WebsocketHandler(echows)) -tas = @schedule WebSockets.serve(server_WS, "127.0.0.1", port_HTTP_ServeWS, false) +tas = @schedule WebSockets.serve(server_WS, "127.0.0.1", port_HTTP_ServeWS) while !istaskstarted(tas);yield();end const servers = [ diff --git a/test/error_test.jl b/test/error_test.jl index f165b79..076ab81 100644 --- a/test/error_test.jl +++ b/test/error_test.jl @@ -23,7 +23,7 @@ info("Start a HTTP server with a ws handler that is unresponsive. Close from cli sleep(1) server_WS = ServerWS( HTTP.HandlerFunction(req-> HTTP.Response(200)), WebSockets.WebsocketHandler(ws-> sleep(16))) -tas = @schedule WebSockets.serve(server_WS, "127.0.0.1", THISPORT, false) +tas = @schedule WebSockets.serve(server_WS, THISPORT) while !istaskstarted(tas); yield(); end sleep(1) res = WebSockets.open((_)->nothing, URL); @@ -39,7 +39,7 @@ server_WS = ServerWS( HTTP.HandlerFunction(req-> HTTP.Response(200)), readguarded(ws_serv) end end); -tas = @schedule WebSockets.serve(server_WS, "127.0.0.1", THISPORT, false) +tas = @schedule WebSockets.serve(server_WS, "127.0.0.1", THISPORT) while !istaskstarted(tas); yield(); end sleep(1) @@ -131,7 +131,7 @@ server_WS = ServerWS( HTTP.HandlerFunction(req-> HTTP.Response(200)), end end end); -tas = @schedule WebSockets.serve(server_WS, "127.0.0.1", THISPORT, false) +tas = @schedule WebSockets.serve(server_WS, "127.0.0.1", THISPORT) while !istaskstarted(tas); yield(); end sleep(3)