Graphical Web browser

[parabyte]

I found this project on github, any use to elks?
https://github.com/jhhoward/MicroWeb

[ghaerr]

A quick look at this Micro Web Browser shows definite potential for ELKS, as it's built for 16-bit real mode and already uses the OpenWatcom C compiler. So getting it running isn't necessarily a huge amount of work, except for the TCP portion.

Getting the project compiled will still need conversion from mTCP, a DOS TCP/IP implementation (different from Watcom's TCP/IP), and that could be a lot of work, but not sure, I have never looked into that. The other major issue is the browser only supports HTTP connections, not HTTPS. These days, I believe there are very very few non-secure websites out there, so the entire project might have very limited usefulness from what any user might naturally expect. Another severe limitation is support for GIF images only, neither PNG nor JPG will display.

[toncho11]

The problem with HTTPS is well known indeed. So people do a special proxy for example between a retro browser and the real website. The proxy does the job of providing each website as http instead of https. The are other hacks where the browser actually presents screenshots and calculates where you click with the mouse, then this click on the image is send as a real click to a real browser and the result is sent back again to the retro bowser. So you have some kind of (slow) refresh rate, but ... you get HTML5 (slow one)! Actually I am sure there are more hacks on the internet, I just do not remember all of them.

https://computernewb.com/wiki/How_to_browse_the_web_on_very_old_computers

[rafael2k]

Considering Arachne is also a possibility, here is the link of a discussion about its port to ELKS:
sraase/arachne-browser#3

[rafael2k]

If we port MicroWeb, may be we use Nano-X instead of directly accessing graphics adapter, keyboard and mouse?

ps: I'm not working on it, it is just a 2025 project TODO. I'd prolly need lots of help with the networking part.
ps2: and concerning a https to http proxy, I could bring up one for ELKS users if there are users interested.

[ghaerr]

If we port MicroWeb, may be we use Nano-X instead of directly accessing graphics adapter, keyboard and mouse?

No, the way Nano-X works is very different - no direct drawing to hardware is allowed, all graphics have to use the Nano-X client API, which are effectively "draw commands" rather than VGA hardware instructions. This would require a rewrite of all MicroWeb graphics.

Regarding keyboard and mouse, I wrote examples/events.c and mouse.c to handle mouse and keyboard event loops, along with evtest.c to show how to use it. This small code has no other dependencies so would be easy to integrate into other projects.

[rafael2k]

The idea for a re-write of the graphic routines - It would be nice to be able to have the browser and a nxterm opened inside Nano-X, for example.

[toncho11]

I tested a bit. With two nxterm opened it is already impossible to edit a text file. In the second nx term I try to edit a file and it prints out of memory error for both edit and vi. So ... there will be no memory left for a browser and nxterm in my opinion.
Except maybe for bank switching or swap file.
I do not know. For example each time a window goes background, it is suspended and saved to disk. Then when clicked in the window manager its memory is loaded again and it resumes.

[ghaerr]

there will be no memory left for a browser and nxterm in my opinion.

I completely agree. Memory is extremely right when running a single shell, Nano-X, nxterm and then vi. On the first port of the modern Nano-X even those three couldn't run together. I'm still working on lowering the sizes of each, but we're now going to be looking at 10-20k reductions, not much more.

For example each time a window goes background, it is suspended and saved to disk. Then when clicked in the window manager its memory is loaded again and it resumes.

Lots and lots of work to get anything like that working, as well as adding swap. I wouldn't think of adding such complexity unless people really started to use ELKS and Nano-X a lot more than now. I've previously thought a lot about adding task swapping to ELKS, and it involves lots of changes, making everything larger for that feature, and then still not much benefit without rewriting the applications. One reason for this is that there is no MMU so the hardware can't tell the OS to "swap in" a page fault when the app needs it - the app must be completely rewritten around bank switching, swapping, etc. The other huge problem is that without an MMU, the application has to be swapped back in to the exact same memory address it originally ran in - very problematic if there are other applications running at that address. It's a huge mess with lots of issues and little benefit, IMO.

[toncho11]

I think a good starting point is https://github.com/ghaerr/elks/blob/master/elkscmd/inet/httpd/httpd.c

The idea is to check how the httpd program makes an http request and then use it in the MicroWeb browser.
Wait a minute ... MicroWeb is in C++, not C!

[toncho11]

Sorry. I was searching for wget tool in ELKS. Is there wget?

[ghaerr]

I was searching for wget tool in ELKS. Is there wget?

Nope. It might be nice to have something like that, to enable grabbing a kernel update or whatever. Of course, actually using the real wget will never work - it's a GNU project, so that means its a total pig, with hopelessly too many source files and RAM usage, not to mention the ridiculous autogen/configure stuff, before even starting a port.

Whether we're talking wget or a browser, IMO the solution for ELKS is to think and search for things that are very, very small and designed for very little memory usage and/or embedded systems.

[toncho11]

It is actually urlget: https://github.com/ghaerr/elks/tree/master/elkscmd/inet/urlget
Isn't it like wget http get ?

[ghaerr]

Isn't it like wget http get ?

Yes, it might be. I've never used it, perhaps play with it and see how it works. The source says it supports FTP and HTTP.

[toncho11]

@ghaerr Is it actually possible to compile the code of MicroWeb which in C++ with OWC for ELKS?

[toncho11]

@rafael2k Here is an example of http get

I imagined this http get to be integrated in the source of MicroWeb.

[rafael2k]

Thanks @toncho11.
There is a net.c, with network abstraction functions for connect and close:
https://github.com/ghaerr/elks/blob/a9a86f2c403e44ff732f66e41b0cf0867fafc622/elkscmd/inet/urlget/net.c

As expected, the code seems very familiar, it is as in Linux as far as I could read. Unless mTCP does crazy stuff, the network port would be smooth I think.

I wonder if there was any recent change in ELKS network code, or all that SO_LINGERing option is still something needed.
I'm checking in the basement for a PC with ISA slot. If I get a 486+ with ISA slot, do you think I can make a NIC (ne2000 may be?) work?

[toncho11]

You can use QEMU with network enabled.
In https://github.com/ghaerr/elks/blob/master/qemu.sh you have a line that enables ne2000 emulation:

NET="-netdev user,id=mynet,$FWD -device ne2k_isa,irq=12,netdev=mynet"

This is the network documentation: https://github.com/ghaerr/elks/wiki#Networking
Actually network card detection is automatic and enabled in the kernel by default, so I am not sure the documentation is 100% up to date.
I think no recompilation is needed currently.

Supported cards are: ne1k, ne2k [ne0], wd8003, wd8013 [wd0] and 3c509 [3c0], so your ne2000 should work.

[rafael2k]

You can use QEMU with network enabled. In https://github.com/ghaerr/elks/blob/master/qemu.sh you have a line that enables ne2000 emulation:

NET="-netdev user,id=mynet,$FWD -device ne2k_isa,irq=12,netdev=mynet"

This is the network documentation: https://github.com/ghaerr/elks/wiki#Networking Actually network card detection is automatic and enabled in the kernel by default, so I am not sure the documentation is 100% up to date. I think no recompilation is needed currently.

Supported cards are: ne1k, ne2k [ne0], wd8003, wd8013 [wd0] and 3c509 [3c0], so your ne2000 should work.

Thanks @toncho11. I can confirm httpget is working as expected here in qemu, and is able to retrieve html content to stdout.

[ghaerr]

As expected, the code seems very familiar, it is as in Linux as far as I could read.

The ELKS networking API is based on early Linux, and is basically BSD sockets with modifications.

Unless mTCP does crazy stuff, the network port would be smooth I think.

It would be a good idea to look and see whether mTCP uses BSD sockets or uses a different API. If not sockets, it could be hard to port. @jhhoward did mention that MicroWeb has a Windows version that uses Winsock; Winsock is pretty close to BSD sockets so that would indeed be a good place to start, if Winsock is currently working on MicroWeb.

I wonder if there was any recent change in ELKS network code, or all that SO_LINGERing option is still something needed.

Unfortunately, the ELKS TCP/IP implementation is complicated, and not fully implemented to spec. Linger is needed in various network daemons and there are also issues with local address reuse with bind and accept, especially when accepting on the same local address repeatedly and reusing sockets within ktcp. The TCP/IP implementation isn't in the kernel, but implemented as a user program ktcp and it is memory constrained with how many sockets/addresses/connections can be open at the same time, etc. That said, things work well for "normal/basic" operations like TCP (UDP and lots of layer 3 are unsupported). Take a look at the network code in MicroWeb and see how much special stuff is required; in general you can probably get something running, but getting it running reliably will take more effort.

Finally, some BSD functionality was too complicated to implement to spec, and the associated routines, like connect or gethostbyname were renamed with an in_ prefix, to in_connect or in_gethostbyname. These implementations all live in libc/net, with the actual system calls like socket, bind, accept etc in the kernel (passed to ktcp).

[toncho11]

Here is a mTCP auto generated sample for retrieving a page for comparison httpget:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <mtcp_api.h>
#include <arpa/inet.h>

#define SERVER "93.184.216.34"
#define PORT 80
#define REQUEST "GET / HTTP/1.1\r\nHost: www.example.com\r\nConnection: close\r\n\r\n"

int main() {
    struct sockaddr_in server_addr;
    mctx_t mctx;
    int sockfd;
    char buffer[4096];

    // Initialize mTCP
    mtcp_init("mtcp_config.conf");
    mctx = mtcp_create_context(0);

    // Create socket
    sockfd = mtcp_socket(mctx, AF_INET, SOCK_STREAM, 0);
    server_addr.sin_family = AF_INET;
    server_addr.sin_port = htons(PORT);
    inet_pton(AF_INET, SERVER, &server_addr.sin_addr);

    // Connect to server
    mtcp_connect(mctx, sockfd, (struct sockaddr*)&server_addr, sizeof(server_addr));

    // Send HTTP request
    mtcp_write(mctx, sockfd, REQUEST, strlen(REQUEST));

    // Receive response
    int bytes;
    while ((bytes = mtcp_read(mctx, sockfd, buffer, sizeof(buffer) - 1)) > 0) {
        buffer[bytes] = '\0';
        printf("%s", buffer);
    }

    // Close socket
    mtcp_close(mctx, sockfd);
    return 0;
}

[rafael2k]

I started a discussion about the MicroWeb ELKS port here: jhhoward/MicroWeb#40

[rafael2k]

While not a graphical browser, bobcat could, with some effort (it is Turbo C, uses PDCurses, etc) be ported to ELKS:
https://github.com/rafael2k/bobcat/

ps: Interesting discussion from 1997 mentioning ELKS: https://lists.nongnu.org/archive/html/lynx-dev/1997-03/msg00426.html

[rafael2k]

I like this from the changelog of bobcat:

- Eliminated the use of floating point variables in Bobcat
  (speedup for PC's without math coprocessors)

: )