Can not use ops with ProxMox 7.x #1356
Comments
|
you have a few different issues here:
if your goal is to go through proxmox we can use this issue for that - now that you have the test.img you should be able to get further along with that - let me know |
|
Thanks for quick reply. Yes, I prefer to use Proxmox instead of other variants, but i need in bridged networking or other solution. P.S. root user i use only for testing. I want create multiple haproxy unikernel machines + several unikernel machines with custom golang app. This golang app working with remote redis servers. For all of this I need a shared local area network, haproxy used for load balancing requests. How do it without bridged networking, i don`t know. Yes I have multiple physical servers in LAN too. Ok lets try to work with ProxMox and onprem test.img: Current config.json: All is empty, nothing errors, in ProxMox web interface no new VMs. |
|
ok - so for promox you should not be trying to set up the networking - that's only if you were trying to run your own infrastructure - the way ops works is that it interacts with the various APIs that clouds and hypervisors such as proxmox export - so instead of you setting up the bridge manually proxmox will do that for you (and manage it) there are 3 env vars that need to be set for proxmox as shown here, https://docs.ops.city/ops/proxmox API_URL, TOKEN_ID, and SECRET - were you able to set those before issuing those commands? also it looks like the image create might've worked - do you see anything new in the storage from the web ui? ops will integrate directly against proxmox's api - you can verify those env vars should be working by grabbing a version like so: |
|
Yes, 3 env vars is set correctly, if not set, ops says error (i temporary removed env vars): Its working, but You say use this url for check - https://ip:8006/api2/json/pve/version - this url gives {"data":null}, In proxmox web interface i didn`t find any images, in proxmox folders same /var/lib/vz too. May be problem in changed urls in late versions of proxmox? Because ops say no errors on every operation and do nothing, like as magic. |
|
so it looks like there might be a hard-coded default this was tested on proxmox-ve 7.1 - i do see that 7.2 is out looking at api docs https://pve.proxmox.com/pve-docs/api-viewer/index.html#/nodes/{node}/qemu it appears that it refers to the cluster node name
that's all currently hard-coded so if you are familiar w/go you could try just substituting or stripping those out and re-building to see if that works for you? if that does work - then we can modify this to use an env var and use that as the backup default? |
|
Ok, i am familiar with go, write any software, give me pls 1-2 days, i come to work and do it, then reply to You. I think additional customizable env var be good solution for different proxmox setups and changeable urls. Thanks, I should have looked at the ops sources myself earlier, just the lack of errors from ops confused me. |
Now I can look images (ops image list -t proxmox):
I try change name in config.json: But every time, when i start ops image create, then ops built image without extension, and then this image can not be uploaded to proxmox. Ok, i hardcoded iso extension in And successfully uploaded image.
Instance is created, but iso image do not connected automatically.
I can take it and improve the сode for work with proxmox, fix a bug with the extension, add custom storage selection CloudConfig option, automatically mount the right image when creating an instance, and I can also make other improvements and features, then I will send a pull request. But what to do with the non-bootable image? I think this is strange - use and upload onprem image as iso image to proxmox. |
|
I figured it out, I was able to boot the image as an attached hard drive based on the virtio driver. On a virtual machine with q35 type. And bridged networking also works with virtio network driver, bridge networking is built by proxmox. root@my:/var/lib/vz/images/103# ping 172.21.1.75 Ok, can i start development improvements(including add more features) to proxmox part code of ops? Then i`ll be send pull requests. I want to write full support of proxmox for ops. |
|
you might be hitting a different issue with the .iso suffix as that looks like it should be getting set here: https://github.com/nanovms/ops/blob/master/proxmox/proxmox_image.go#L67 the image can't be ran as a cdrom - when you create the instance with ops it attaches as a virtio disk https://github.com/nanovms/ops/blob/master/proxmox/proxmox_instance.go#L94 as you found out - ops will set up all of this for you - the intention is that the end-user shouldn't be doing anything manually at all yes if you want to work on this just drop each feature/bug into a sep. pr please - for instance the default node name set by env var could be one |
|
i fixed the .iso extension issue - at some point in the past half year or so we had removed all extensions which broke that but proxmox requires it; also for whatever reason we never put in code to mv the 'iso' to disk so put that in as well - #1362 |
|
Thanks, later i look into api of proxmox and test all of possibilities with images (including upload, replace, attach, etc), because in proxmox normally need to create .raw images, not upload .iso . But let's see later what options are possible. |
|
this #1363 fixes the case where an image isn't built locally yet before trying to upload to proxmox - it ensures the image is built first which is the default behavior for all other targets |
|
I created new pull request with additional checks of returned empty data from proxmox api: Later, when i will add configurable storage names, then I will add check for errors for this two calls functions: |
|
I made configurable storages, bridge interface(with check of configured bridge in proxmox) via CloudConfig and config.json. I created a config.CloudConfig.BridgeName0 (Not BridgeName), because later I think we need to have support of minimum two interfaces (if nanos and ops have planned support of more than one connected network interface). For example in my case I need in the future have ability to use two bridge interfaces from ProxMox in case of using haproxy in nanovms (first for internet, second for local area network and load balancing/high availability to backend apps in local area network). Also I added instanceName from config.RunConfig.InstanceName (You made this in RunConfig part of config.json) and now can be created instances with timestamp support from cmd/cmd_instance.go. I do not want broke any backward-compatibility, and uses config.RunConfig.InstanceName as is. In general, I need to know where to place which options(in RunConfig/CloudConfig) for ProxMox (StorageName, IsoStorageName, BridgeName0, CPUs, Cores, Memory, Onboot, Machine Type, etc) Can I create a pull request? Thanks. In which case, I can always rewrite code. Example of code: Example of config.json |
|
I think, config for real production in case of ProxMox is will be like this (minimum): |
|
so some thoughts (as someone who doesn't work day to day with proxmox):
|
|
Hello, I try to use ops with local ProxMox 7.2 and Ops 0.1.32, and it isn`t working:
System information:
Access rigths is full on token in ProxMox.
Image can not be build, nothing any errors from ops.
In onprem mode is all starting ok, excluding bridge - is non-functional:
Machine is not available through host bridge.
I try to add "TapName": "tap0" or "TapName": "tap75i0" or any other name, but in this case of use onprem I have additional trouble (Not able to create tap, but modprobe tap is ok, and on this server I have so much other tap* interfaces):
Only all working through exposed port. Without TapName,Bridged,BridgeName,IPAddress,Gateway,NetMask options:
But this is can not be used for production.
Please, can anyone help? May be i missing something or do something wrong? Thanks.
The text was updated successfully, but these errors were encountered: