reynir/albatross
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity
305 commits 3 branches 0 tags 791 KiB
Commit graph

159 commits

Author SHA1 Message Date
Hannes Mehnert 4d3bb777e1 vmm_vmmd: introduce and use type 'a create 2019-01-20 22:09:33 +01:00
Hannes Mehnert e58ab236b7 vmmd: lock self_destruct with a mutex, respect result of killall (nothing killed, nothing to do) 2019-01-20 22:09:33 +01:00
Hannes Mehnert d4e31da27f vmmd: setup and teardown stat by create continuation, and vmm_vmmd.handle_shutdown 2019-01-20 22:09:33 +01:00
Hannes Mehnert fdcea94a0b vmmd: store waiter, not tasks in Vmm_vmmd.t -- create task and waiter on demand (destroy / create --force), instead of at each unikernel creation 2019-01-20 22:09:33 +01:00
Hannes Mehnert fc63a89429 Vmm_unix: use Sys.sigterm (and rely on caml_convert_signal_number to be cross-platform) 2019-01-20 20:37:30 +01:00
Hannes Mehnert dfd22be62b Vmm_unix: don't reset image of a config 2019-01-20 20:35:55 +01:00
Hannes Mehnert accb21b8b7 Vmm_unix.exec: rename vm to config 2019-01-20 20:35:26 +01:00
Hannes Mehnert d30e118c76 Vmm_core.Name: use 'vm:' instead of 'name' as literal string to print 2019-01-20 20:31:40 +01:00
Hannes Mehnert 15e1f5ecf6 vmm_unix: ensure lazy uname being called only once 2019-01-20 20:28:51 +01:00
Hannes Mehnert 7b8f8fafbd close fd early, no need to carry file descriptors around 2019-01-20 17:43:44 +01:00
Hannes Mehnert 094922f6b0 vmmd: teardown gracefully (kill all vms), install SIGTERM handler to teardown 2019-01-18 01:14:11 +01:00
Hannes Mehnert 58bd77bc5f stats: pass bridge device through, vmm device name as well to allow arbitrary bhyve statistics, vmmc_local: add stats_add and stats_remove subcommands 2019-01-15 00:25:59 +01:00
Hannes Mehnert fd4a5a5e22 inspect kinfo_proc structure for information about: 
- virtual size
- resident size (in pages)
- text size (in pages)
- data size (in pages)
- stack size (in pages)

develop independent vmmc_stat for testing (not installed)
 2019-01-06 01:12:56 +01:00
Hannes Mehnert 030f5aa379 vmm_unix, create_block: first create file, and then truncate 2018-12-07 00:07:16 +01:00
Hannes Mehnert bda342f136 re-support parsing of AV2 log entries, document path forward strategy 2018-12-06 22:55:13 +01:00
Hannes Mehnert e28ea84548 Log: new event `Hup 2018-12-06 22:53:15 +01:00
Hannes Mehnert b5a068555c vmmd_tlS_inetd 2018-11-23 21:07:36 +01:00
Hannes Mehnert c8f1030403 rename Vm to Unikernel 2018-11-13 01:02:05 +01:00
Hannes Mehnert 85372b0c7e rework resources: now block, vms, and policies are in separate tries 2018-11-13 00:06:43 +01:00
Hannes Mehnert b5c9cdea6a cleanups 2018-11-12 22:19:39 +01:00
Hannes Mehnert dec32e6247 bump wire version 2018-11-12 22:11:06 +01:00
Hannes Mehnert 8ccda0e410 refactor bridge: use a string instead of a complicated thing 2018-11-12 22:07:45 +01:00
Hannes Mehnert 2e7f2730a2 move Vm to submodule 2018-11-11 03:24:50 +01:00
Hannes Mehnert 561ba5c5df put Policy in a submodule 2018-11-11 03:09:37 +01:00
Hannes Mehnert 89a1d30154 cleanups in respect to directories and scope 2018-11-11 02:33:00 +01:00
Hannes Mehnert 43379d6d9d rename Vmm_core.id to Vmm_core.Name.t and make it private - also check constructors to fit into 20 chars ldh (and in Vmm_tls max depth = 10) 2018-11-11 01:44:31 +01:00
Hannes Mehnert 6dcde8eb68 block device support 2018-11-11 00:01:56 +01:00
Hannes Mehnert 6945d21422 Vmm_core.drop_super / is_sub_id: fix function (used to assume reverse ordered labels) 2018-11-09 01:27:22 +01:00
Hannes Mehnert 75372a792f fix resource policies. it was checking too many vms: 
vm foo.bar is active with 32mb
add_policy bar --mem 16 <- failed :/

what is checked on add_policy <id> <new-policy>?
- all policies above <id> that <new policy> is a sub-policy
- all policies below <id> that each is a sub-policy of <new-policy>
- resource usage of vms below <id> is within <new-policy> limits (number of vms, memory, network access, cpuids)
 2018-11-03 00:05:10 +01:00
Hannes Mehnert 79068c8abf error logging on error in handle_command 2018-11-03 00:04:47 +01:00
Hannes Mehnert 0c58ebeedf lower debug level for end of file 2018-11-01 01:54:10 +01:00
Hannes Mehnert 9f674f7e6f forgot loop on this exit in add_policy 2018-11-01 01:23:45 +01:00
Hannes Mehnert 7c34c61d43 vmmd_tls needs looping behaviour after adding a policy to start vm 2018-10-31 23:03:30 +01:00
Hannes Mehnert c669be8e02 address most of @cfcs comments 2018-10-29 17:14:51 +01:00
Hannes Mehnert a124b3eb30 upgrade to recent decompress 2018-10-29 00:05:55 +01:00
Hannes Mehnert 2b85c65dd8 minor fixes from testing: do not require vm to be present for force-create, fix id generation in vmm_tls, use 32mb memory for unikernels by default 2018-10-28 23:06:15 +01:00
Hannes Mehnert 9191d2cf9a drop version AV0, AV1; refactor vmm_asn 2018-10-28 22:52:20 +01:00
Hannes Mehnert 0f9375dc29 use oid 42 again 2018-10-28 22:39:31 +01:00
Hannes Mehnert 947b82f4f0 vmm_tls: ensure that add_policy commands carry a non-empty name 2018-10-28 22:30:08 +01:00
Hannes Mehnert 34291dbe65 vmmp_request 2018-10-28 22:14:39 +01:00
Hannes Mehnert 40519afbb7 issue policy_add commands by vmmd_tls for certificate chain 2018-10-28 20:50:10 +01:00
Hannes Mehnert 8ab37d6b3b resources: remove_vm and remove_policy - no need to intertwine into a single remove 2018-10-28 19:50:48 +01:00
Hannes Mehnert 7b8f2cf802 add policy does nothing when received policy is equal to stored one 2018-10-28 19:41:06 +01:00
Hannes Mehnert 296b7a9b01 vmmd_tls: close sockets appropriately 2018-10-28 19:19:38 +01:00
Hannes Mehnert 5e921d7345 skip empty common names in vmm_tls 2018-10-28 19:04:24 +01:00
Hannes Mehnert 8f02d8263d wip: vmmc_bistro 2018-10-28 02:03:27 +02:00
Hannes Mehnert a60f866f70 fewer lists, read replies (to sockets) in vmmd 2018-10-26 21:30:54 +02:00
Hannes Mehnert 01f933702d move stuff around 2018-10-26 21:30:54 +02:00
Hannes Mehnert 85a507db54 whitelist commands accepted via tls certificate 2018-10-26 21:29:59 +02:00
Hannes Mehnert 992e1b0a2b - Vmm_ring is now polymorph (alows to store log_entry :D) 
- Vmm_console/log/stats do not read multiple times
  console_add loops
  console_subscribe terminates (a stream of messages is sent)
  log data stream loops
  log_subscribe terminates (a stream of data is sent)
  stat_add loops
  stat_remove loops
  stat_subscribe terminates (a stream of stats is sent)
terminates means: reads once more, and closes socket after second read returned
loop processes further incoming data
 2018-10-26 21:29:59 +02:00
Hannes Mehnert b55281d1e5 include version in log_entries on disk, read log file on startup (and write events to ring store) 2018-10-26 21:29:59 +02:00
Hannes Mehnert cdae37b0bf interface for vmm_ring 2018-10-26 21:29:59 +02:00
Hannes Mehnert 04367421bf since argument for log_subscribe and console_subscribe 2018-10-26 21:29:59 +02:00
Hannes Mehnert 89fea934a7 copyright 2018-10-26 21:29:59 +02:00
Hannes Mehnert a064c7f58e move more stuff around 2018-10-26 21:29:59 +02:00
Hannes Mehnert d513269453 move stuff into vmm_commands 2018-10-26 21:29:59 +02:00
Hannes Mehnert 6f18f1bfff type data for streamed thingies 2018-10-26 21:29:59 +02:00
Hannes Mehnert ce0c42fa77 more cleanups 2018-10-26 21:29:59 +02:00
Hannes Mehnert 46548418cd minor cleanup: stats type 2018-10-26 21:29:59 +02:00
Hannes Mehnert f3c67f626a more cleanups 2018-10-26 21:29:59 +02:00
Hannes Mehnert d896d89bba . 2018-10-26 21:29:59 +02:00
Hannes Mehnert f5ce2d8826 reuse commands from Vmm_asn.wire_commands for certificates 2018-10-26 21:29:59 +02:00
Hannes Mehnert d6c87bacde minor tweaks 2018-10-26 21:29:59 +02:00
Hannes Mehnert a08f35ee5e cleanups 2018-10-26 21:29:59 +02:00
Hannes Mehnert 811f3abc50 adjustments 2018-10-26 21:29:59 +02:00
Hannes Mehnert c399501a18 get rid of vm_config.vname 2018-10-26 21:29:59 +02:00
Hannes Mehnert 0441b8ab25 tls endpoint 2018-10-26 21:29:59 +02:00
Hannes Mehnert f939ff5a58 influx stats 2018-10-26 21:29:59 +02:00
Hannes Mehnert 1d4d7509dc remove vmm_wire, use asn.1 2018-10-26 21:29:59 +02:00
Hannes Mehnert 51a0344477 fix warnings 2018-10-26 21:29:59 +02:00
Hannes Mehnert 2239aafdb7 revive vmm_client 2018-10-26 21:29:59 +02:00
Hannes Mehnert bcb280aa00 refactor commands into vmm_commands 2018-10-26 21:29:59 +02:00
Hannes Mehnert efc043cd5c fix 'vmmc info' 2018-10-26 21:29:59 +02:00
Hannes Mehnert 182e2ae10c policies: 
vmmc now has more subcommands
  - policy [-n name] returns all policies in name and below
  - add_policy [-n name] [--cpu cpuid] [--mem mem] [--bridge bridge] [--block size] adds a policy
  - remove [-n name] removes policy at name

policy is just the same which is in vmm_req_delegation, and vmm_resources now check them:
- you cannot insert a subpolicy violating the prefix
- you cannot insert a policy which would forbid current resource usage
- you cannot insert a policy with which any subpolicy would be invalid
- you can adjust (increase/decrease) a policy if the above invariants are kept

implement "force create" directly in vmmd: much nicer to
 - check resource constraints,
 - kill vm potentially,
 - and create a new vm,
all as single transaction.
 2018-10-26 21:29:59 +02:00
Hannes Mehnert ea83013068 delegation -> policy 2018-10-26 21:29:59 +02:00
Hannes Mehnert e413b8c99a remove naming struggle in vm_config and Log.hdr 2018-10-26 21:29:59 +02:00
Stefan Grundmann 4c5a795a3b console fifos in separate directory 2018-10-07 01:24:33 +00:00
Stefan Grundmann 133884faf4 log, stats and console socket go in their own directory 2018-10-07 00:32:25 +00:00
Hannes Mehnert 38094a53e3 use vmm_trie in log and stat, cleanups 2018-09-28 22:44:38 +02:00
Hannes Mehnert 02f8d94db8 s/ukvm/hvt/ 2018-09-21 22:31:04 +02:00
Hannes Mehnert 38b98ab318 minor 2018-09-20 23:19:55 +02:00
Hannes Mehnert 99ba1c5e4b stats are back now! no longer two pullers, but now with one pusher :) 2018-09-20 22:53:42 +02:00
Hannes Mehnert e7b4742964 less is more, also unify default socket paths 
and vmmc console command
 2018-09-19 21:53:18 +02:00
Hannes Mehnert bd10209297 wip, vmmc and vmmd talk with each other! 2018-09-09 20:52:04 +02:00
Hannes Mehnert 9ec69e23cc rename Vmm_commands to Vmm_unix 2018-07-07 23:14:49 +02:00
Hannes Mehnert c9afb6f5dc CLOEXEC for all the openfile() calls to avoid leaking the fifo's (created and opened by vmmd itself) to ukvm-bin started later 2018-04-30 00:20:28 +02:00
Hannes Mehnert 2bf1b96178 get rid of FreeBSD kludge by using devfs! 2018-04-26 22:17:54 +02:00
Hannes Mehnert 0583fbfaf1 stats: transmit vmid in add/remove/stats, pid only in add 
don't use /tmp anymore, but /var/run/albatross for fifos + sockets + vm images,
  and /var/db/albatross for ukvm-bin and crls, and /var/log/albatross for logging

vmm_console/vmm_log/vmm_stats_lwt: delete socket on startup if it exists

vmm_influxdb_stats: connects to vmm_stats socket and pushes every interval in
 influxdb line format via tcp to specified host and port
 2018-04-26 00:03:46 +02:00
Hannes Mehnert 9696953cd7 revise force-restart: now with wait for kill and resource cleanup before start 
allows to cleanup various hacks, such as checking for pid in vmm_resources
or removing temporarily the allocated resources from the resource map in vmm_engine

semantics is now slightly different, but for sure enhanced.
- each VM has a Lwt.wait () task attached in Vmm_engine.t (tasks : 'c String.Map.t)
- normal create shouldn't be much different, apart from memoizing the sleeper
- after waitpid is done in vmmd, and vmm_engine.shutdown succeeded, Lwt.wakeup is called for the sleeper
- force create now:
 - checks static policies
 - looks for existing VM (and task), if present: kill and wait for task in vmmd
 - continue with presence checking of vm name, dynamic policies, allocate resources (tap, img, fifo)

this means the whole randomness in filenames can be removed, and the
communication between vmm_console and vmm_client is working again (attach/detach
could not work since vmm_console knew only about "albatross.AAA.BBB.RANDOM",
whereas vmm_client insisted on "AAA.BBB"

resource overcommitment (and races in e.g. block device closing + opening) are
gone now, only if the old vm is cleanup up, resources for the new one are
allocated and it is executed
 2018-04-05 01:02:45 +02:00
Hannes Mehnert a89b2925fd Vmm_core.cmd is now a variant (no longer polymorphic variant), some renames in Vmm_wire.Stats and Vmm_wire.Console to disambiguate 2018-04-04 22:16:31 +02:00
Hannes Mehnert fdab43aed6 vmm_engine: finish renaming Destroy_image -> Destroy_vm (broken since bb61388cfc) 2018-04-04 21:46:37 +02:00
Hannes Mehnert 7a4661b2e1 style: require lwt 3.0.0, fix warnings, disable 4 (fragile pattern matching) and 48 (implicit elimination of optional argument) 2018-04-03 22:58:31 +02:00
Hannes Mehnert a0c0f39734 vmm_stats: more debug, ignore vmmapi_open failure 
vmm_stats_lwt: drop all pids on socket disconnect
vmmd: setup statistics slightly later (after the chmod on FreeBSD)
 2018-04-01 23:59:12 +02:00
Hannes Mehnert c978dcfb14 less debug output of resources 2018-03-22 23:30:15 +01:00
Hannes Mehnert e48d3b3457 vmm_lwt/vmm_tls: less debug output of read/write stuff 2018-03-22 17:00:08 +01:00
Hannes Mehnert d3941e70c6 style 2018-03-22 17:00:08 +01:00
Hannes Mehnert b9d5fa94f9 call waitpid() earlier, separate freebsd kludge (chmod g+rw), fixes #9 2018-03-22 17:00:08 +01:00
Hannes Mehnert db8ae1ee37 compression, fixes #6 2018-03-22 17:00:08 +01:00
Hannes Mehnert cfa7ccd1e0 safer and clearer error semantics for all processes, fixes #5 2018-03-22 17:00:08 +01:00
Hannes Mehnert 88012094f8 remove unused Vmm_commands.waitpid, catch EINTR in call to Lwt_unix.waitpid (in Vmm_lwt) 2018-03-22 17:00:08 +01:00