add nerf viewer

[WYK96]

add nerf viewer client and server.

[CLAassistant]

All committers have signed the CLA.

[GACLove]

1. Add Architecture diagram in README.md
2. Delete the top-level directory "nerf_viewer"
3. Rename the directory "nerf_viewer" to "viewer":
4. Rename the directory "bridge_server" to "server"
5. Write a start.sh or start.py script to launch the server and client
6. Generate a default .gitignore file, visit https://www.toptal.com/developers/gitignore.
7. To delete unused files and modules

[yl-1993]

"###" would be enough

[WYK96]

Sorry for the typo. The title hierarchy issue has been fixed.

[yl-1993]

Shall we replace it with new vierwer screenshot? The coordinate axis on upper right corner is tilted.

[WYK96]

The original version of README.md is for debug purpose. In the lastest doc, the screenshots have been removed.

[yl-1993]

Nerf -> NeRF

[WYK96]

Glitch fixed: Nerf Viewer -> XRNeRF Viewer

[yl-1993]

Shall we rename it to bridge_server.py? As there is no conflict, it is not necessary to add _subprocess suffix.

[WYK96]

1. Renamed bridge_server_subprocess.py to bridge_server.py
2. Renamed function run_bridge_server_as_subprocess to start_bridge_server

[yl-1993]

npm start

Same as L17, L22 and L32

[WYK96]

This README.md is automatically generated by Create React App. It has been replaced to XRNerf Viewer.

[yl-1993]

For Learn More section, as each subsection. only as one sentence, shall we use bullet point directly? For example,

• Code Splitting
• Analyzing the Bundle Size
• ...

[WYK96]

This README.md is automatically generated by Create React App. It has been replaced to XRNerf Viewer.

[yl-1993]

Remove commented content if they are unnecessary anymore.

[WYK96]

Unused code blocks removed.

[yl-1993]

1. Move two readme files to ${ROOT}/docs/en/visualization/, e.g., nerf_viewer.md, so it can also be viewed viareadthedocs.
2. Move all png to ${ROOT}/resources/viewer/ and try to compress the single filesize under 10KB.

[maoXyzt]

• format *.py files via pre-commit hooks according to the project's .pre-commit-config.yaml
• add one blank new line to the end of code files if there is not one.

[LazyBusyYang]

Do we have any installation instruction in English? Say https://github.com/nvm-sh/nvm#installing-and-updating.

[WYK96]

The original version of README.md is for debug purpose. In the lastest doc, the installation instructions has been updated so that it is suitable for non-chinese users.

[LazyBusyYang]

We have a requirements folder in the root directory of the repository, where many different requirements files are named according to their functionality and placed here, such as requirements/synbody.txt. You can also consider renaming your requirements.txt to something like bridge_server.txt. In addition, most of the relative paths in our documentation are relative to the root directory of the repository, not to subdirectories like viewers.

[WYK96]

For requriements, the dependencies list of server has been moved from
python/xrprimer/services/xrnerf/requirements.txt to requirements/service_xrnerf.txt.

For paths, they are now relative to the root directory of the repository, e.g., ./run.py to xrprimer/python/xrprimer/services/xrnerf/run.py.

Appreciate for the suggestion.

[LazyBusyYang]

Suggested change

	python ./run_viewer.py
	python .python/tools/un_viewer.py

We often put files with a main function program entry point in the tools folder to distinguish them from files that are only called and not used as entry points.

[LazyBusyYang]

What if we merge this file with the .gitignore file in the root directory?

[WYK96]

The original .gitignore file is for debug purpose only. It has been removed now.

[LazyBusyYang]

Suggested change

	def get_free_port(default_port: int = None):
	from typing import Union
	def get_free_port(default_port: Union[int, None] = None):

[WYK96]

The default_port: int has been declared as default_port: Optional[int].

[LazyBusyYang]

For users, there are two possible scenarios. In the first scenario, the user does not care which port is used as long as the service is started successfully and the port used is returned. The current function can meet this requirement. In the second scenario, the user may be required to use a specific port, such as port 80. If the target port is occupied, an exception should be thrown and there is no need to continue running. Considering these two scenarios, we can modify the parameter list of this function, for example:

1. def bind_port(allow_random_port: bool, preferred_port: Union[int, None] = None) -> int:
2. def bind_port(target_port: Union[int, None] = None) -> int:

In the first example, the allow_random_port parameter is used to determine which usage scenario to use, and preferred_port is the port number that is preferentially attempted. In the second example, the target_port parameter is used to determine the usage scenario. If it is provided, the target port must be used. If it is not provided, there are no requirements and any available port can be returned.

[WYK96]

Updated port binding logics: if the user specified a port, the server will check whether the port is already in use and then create connections using zmq, otherwise the server will automatically find an available port.

[LazyBusyYang]

ChatGPT:
In Python, Union[int, None] and Optional[int] are similar in that they both indicate that a variable can be either an integer or None. However, there is a subtle difference between them.

Union[int, None] explicitly specifies that the variable can be either an integer or None. It does not allow any other type to be assigned to it. On the other hand, Optional[int] is actually an alias for Union[int, None], which means it represents the same set of possible types. However, Optional[int] does not explicitly indicate that the variable cannot be assigned any other type.

Therefore, using Union[int, None] is more specific and explicit, and may help prevent type-related bugs in your code. However, both Optional[int] and Union[int, None] can be used interchangeably in most cases.

[maoXyzt]

I doubt the response from ChatGPT. According to PEP484 which is the official definitions of python typing hints, Optional[...] is no more than a shorthand of Union[..., None]. It says in The typing module section:

Optional, defined by Optional[t] == Union[t, None]

So do the documentation of typing.Optional:

Optional[X] is equivalent to X | None (or Union[X, None]).

It's also recommended to use Optional[...] for readability. Since for complex types, we could combine Union with Optional. For example, from def foo(x: Optional[Union[str, pathlib.Path]] = None): ..., we know that foo accepts a path-like argument or None as default. And we can extract this complex type to be PathLike = Union[str, pathlib.Path] and def foo(x: Optional[PathLike] = None): .... Otherwise, we need two typings of Union[str, pathlib.Path, None] and Union[str, pathlib.Path].

There is a related disccusion at StackOverflow says why Optional is suggested:

https://stackoverflow.com/questions/51710037/how-should-i-use-the-optional-type-hint

[maoXyzt]

from ChatGPT:

In python, is there any difference between Optional[int] and Union[int, None]`?

Yes, there is a difference, but it's mainly in how they are expressed. Both Optional[int] and Union[int, None] represent the same type, which is either an integer or None. The difference lies in how you express this type annotation.

Optional[int] is a more concise and readable way to express a type that can either be an integer or None. It is equivalent to Union[int, None]. The Optional type is just a shorthand for a Union with None as one of the alternatives.

In Python, you can use both Optional[int] and Union[int, None] to annotate a variable or a function argument that accepts either an integer or None. However, it's generally recommended to use the Optional type for readability and simplicity.

from typing import Optional, Union

def example_optional(x: Optional[int]) -> None:
    print(x)

def example_union(x: Union[int, None]) -> None:
    print(x)

example_optional(42)  # Prints 42
example_optional(None)  # Prints None
example_union(42)  # Prints 42
example_union(None)  # Prints None

In both cases, the functions example_optional and example_union accept either an integer or None as their argument, and both function calls produce the same output.

---

And I asked ChatGPT about the practical difference in static type checkers mypy.

Is there any difference between Optional[int] and Union[int, None] when using python type checkers like mypy?

No, there is no difference between Optional[int] and Union[int, None] when using Python type checkers like mypy.

Optional[int] is actually a shorthand for Union[int, None], and they are equivalent in terms of type checking. Both indicate that the annotated variable can either be an integer or None.

Using Optional[int] is generally preferred as it is more concise and easier to read. However, if you are using an older version of Python or a type checker that does not support Optional, you can use Union[int, None] instead.

[WYK96]

Since Optional[int] is equivalent to Union[int, None] whereas more concise and readable, we're using Optional as the type annotation.

[LazyBusyYang]

0.0.0.0 may allow access to a larger range than 127.0.0.1. Should we change the default value to 0.0.0.0?

[WYK96]

Replace the default ip address to 0.0.0.0 makes zmq fail to connect with client.

TODO: figure out why this happens.

[LazyBusyYang]

process in cleanup is passed by arg list, but in poll_process it looks like a global var.
Considering that the process variable has a clear lifetime and these functions are strongly related to it, would it be better to rewrite this code as a class and access it through self.sub_process?

[LazyBusyYang]

Same as https://github.com/openxrlab/xrprimer/pull/40/files#r1191927947

[LazyBusyYang]

What about not as cv? We use cv2 directly in other modules inside XRPrimer.

[WYK96]

Changed cv to cv2. The OpenCV is used to generate fake images for debug purpose, which should be further replaced by the nerf backend.
TODO: remove the dependency once the nerf backend is ready.

[LazyBusyYang]

Please check logger example in class.
self.logger.error(msg) logs the message in error level, if a file handler has been configured, the message will never be lost. After logging, by using raise ValueError, we can prevent the program from continuing to run and causing unexpected errors, which can make it easier for developers to identify the location where the error occurred.

[LazyBusyYang]

Suggested change

	"""fetch data from state"""
	# fetch data from state

pre-commit will find everything breaks rules.

[WYK96]

The issue has been fixed by pre-commit.

[LazyBusyYang]

Why 0.05? It looks like 20 fps. If the time parameter for sleep is a user-specified parameter that is set during construction or dynamically determined based on the program's running state, it may be better.

[WYK96]

The relief time is a time step that the backend sleep for some time to avoid websocket blocking. The value of it needs to be further evaluated. A more reasonable relief time would be:

$relief\_time = time\_step - (backend\_inference\_elapsed + viewer\_render\_elapsed)$

e.g. If we want the viewer renders at 60fps, our per-frame budget is 16.6ms. Assuming that the nerf inference time is 5ms and viewer render latency is 4ms, the relif time is 16.6-5-4=7.6ms

Unfortunately, we do not have a really available backend up to now.

TODO: replace to a more reasonable relief time once the nerf backend is ready.

[LazyBusyYang]

Same as https://github.com/openxrlab/xrprimer/pull/40/files#r1191947804.

[LazyBusyYang]

In most cases, we do not use Exception because it corresponds to too broad of an error category. If we can identify a specific type of error, such as RuntimeError or ValueError, we should only catch the known and specific types of errors.

[LazyBusyYang]

Same as https://github.com/openxrlab/xrprimer/pull/40/files#r1191930152

[LazyBusyYang]

Is it possible to redirect these logs to a designated path, such as logs/npm_log.txt?

[LazyBusyYang]

If these lines does not work, and it is no important comment, please remove them from official commit. To keep them, push another branch in a forked repo.

[WYK96]

Unused code blocks removed.

[maoXyzt]

NVM is not a requirement but a recommendation.

Will it be better for users to choose their own method to install node.js?

And NVM could be an optional choice like:

Install node.js 18.15.

It's suggested to use `NVM` to install and manage `node.js` installations.

[WYK96]

The original version of README.md is for debug purpose. In the lastest doc, the installation guide has been replaced to:

The package.json contains dependencies to build the viewer, which can be installed using Node Package Manager(npm). It's suggested to manage Node.js installations with Node Version Manager(nvm). The installation instructions of nvm can be found here.

[maoXyzt]

This a blog in Chinese not suitable for non-Chinese users and only for windows platform. For i18n, post official installation instructions.

[WYK96]

The original version of README.md is for debug purpose. In the lastest doc, the link for nvm installation guidelines has been replaced from https://juejin.cn/post/7120267871950733320/ to https://heynode.com/tutorial/install-nodejs-locally-nvm/.

[maoXyzt]

nodejs 18.15 could be precise enough. Don't pin the patch version.

[WYK96]

The patch version has been removed: 18.15.0 -> 18.15

[maoXyzt]

Pin tornado's major version to prevent incompatibility (i.e. tornado>=6,<7), since a major version update could probably break current codes.

[maoXyzt]

Are args of List[str] & kwargs of Dict[str, str]?
Since they are not used, their typings could be omitted.

[maoXyzt]

imports unsorted.

Recommended to run isort to update all bridge_servers python files.

[WYK96]

All python imports have been sorted by isort in pre-commit.

[maoXyzt]

This file includes only commented contents

[WYK96]

Unused file removed.

[maoXyzt]

This file includes only commented contents

[WYK96]

Unused file removed.

[maoXyzt]

I doubt the response from ChatGPT. According to PEP484 which is the official definitions of python typing hints, Optional[...] is no more than a shorthand of Union[..., None]. It says in The typing module section:

Optional, defined by Optional[t] == Union[t, None]

So do the documentation of typing.Optional:

Optional[X] is equivalent to X | None (or Union[X, None]).

It's also recommended to use Optional[...] for readability. Since for complex types, we could combine Union with Optional. For example, from def foo(x: Optional[Union[str, pathlib.Path]] = None): ..., we know that foo accepts a path-like argument or None as default. And we can extract this complex type to be PathLike = Union[str, pathlib.Path] and def foo(x: Optional[PathLike] = None): .... Otherwise, we need two typings of Union[str, pathlib.Path, None] and Union[str, pathlib.Path].

There is a related disccusion at StackOverflow says why Optional is suggested:

https://stackoverflow.com/questions/51710037/how-should-i-use-the-optional-type-hint

[maoXyzt]

from ChatGPT:

In python, is there any difference between Optional[int] and Union[int, None]`?

Yes, there is a difference, but it's mainly in how they are expressed. Both Optional[int] and Union[int, None] represent the same type, which is either an integer or None. The difference lies in how you express this type annotation.

Optional[int] is a more concise and readable way to express a type that can either be an integer or None. It is equivalent to Union[int, None]. The Optional type is just a shorthand for a Union with None as one of the alternatives.

In Python, you can use both Optional[int] and Union[int, None] to annotate a variable or a function argument that accepts either an integer or None. However, it's generally recommended to use the Optional type for readability and simplicity.

from typing import Optional, Union

def example_optional(x: Optional[int]) -> None:
    print(x)

def example_union(x: Union[int, None]) -> None:
    print(x)

example_optional(42)  # Prints 42
example_optional(None)  # Prints None
example_union(42)  # Prints 42
example_union(None)  # Prints None

In both cases, the functions example_optional and example_union accept either an integer or None as their argument, and both function calls produce the same output.

---

And I asked ChatGPT about the practical difference in static type checkers mypy.

Is there any difference between Optional[int] and Union[int, None] when using python type checkers like mypy?

No, there is no difference between Optional[int] and Union[int, None] when using Python type checkers like mypy.

Optional[int] is actually a shorthand for Union[int, None], and they are equivalent in terms of type checking. Both indicate that the annotated variable can either be an integer or None.

Using Optional[int] is generally preferred as it is more concise and easier to read. However, if you are using an older version of Python or a type checker that does not support Optional, you can use Union[int, None] instead.

[WYK96]

1. Move two readme files to ${ROOT}/docs/en/visualization/, e.g., nerf_viewer.md, so it can also be viewed viareadthedocs.
2. Move all png to ${ROOT}/resources/viewer/ and try to compress the single filesize under 10KB.

1. Moved client and server readme files to ${ROOT}/docs/en/web_renderer/xrnerf and ${ROOT}/docs/en/services/xrnerf, repectively.
2. Moved helper images to ${ROOT}/resources. Since the original version of README.md is for debug purpose, the screenshots have been removed in the lastest doc.

[WYK96]

1. Add Architecture diagram in README.md
2. Delete the top-level directory "nerf_viewer"
3. Rename the directory "nerf_viewer" to "viewer":
4. Rename the directory "bridge_server" to "server"
5. Write a start.sh or start.py script to launch the server and client
6. Generate a default .gitignore file, visit https://www.toptal.com/developers/gitignore.
7. To delete unused files and modules

1. Added a framework figure in ${ROOT}/docs/en/web_renderer/xrnerf/README.md
2. The top-level directory nerf_viewer has been deleted. Now the viewer client is on ${ROOT}/web_renderer/xrnerf and the viewer server is on ${ROOT}/python/xrprimer/services/xrnerf
3. The directory ${ROOT}/viewers/nerf_viewer/client is renamed to ${ROOT}/web_renderer/xrnerf
4. The directory ${ROOT}/viewers/nerf_viewer/bridge_server is renamed to ${ROOT}/python/xrprimer/services/xrnerf
5. Added ${ROOT}/web_renderer/xrnerf/start.bat to launch the client and ${ROOT}/python/xrprimer/services/xrnerf/run.py to launch the server.
   TODO: test the framework on Linux systems and add a start.sh for client startup
6. TODO: it seems that https://www.toptal.com/developers/gitignore does not provide an option to generate JavaScript and TypeScript .gitignore file. Need to be fixed later.
7. Deleted redundant files and modules:
   • Removed redundant redux stored variables
   • Removed unused code blocks

[WYK96]

• format *.py files via pre-commit hooks according to the project's .pre-commit-config.yaml
• add one blank new line to the end of code files if there is not one.

1. Reformatted python scripts with ${ROOT}/.pre-commit-config.yaml utilizing pre-commit hooks
2. For each python script, added a new line at the end of file