One thing I can have is instead of PNGs for images, I can have a kind of vector format. SVG is complex and requires XML parsing however.
One thing I need to consider is that with the register allocator if there is no state then I must store it into a jump target or restore it if it exists.
I believe for the stack map state, I will need a local cache marker as such. Basically assign a unique value to a local and then for stack entries have it so it points to that local and store the unique value. When a local value is written to then the unique value is changed. Although really the opposite must occur. Local variables need to have a link to the stack items which contain a copy of the local. In the event that a local variable changes, any existing stack entries are to be copied via generated code. Across jump target bounds, I should clear the entries so their cached state is removed since it might not make much sense. There can be a jump from two places which has the same stack layout but where the value is sourced from two completely different locals. Then when a local value wants to change to another value, that check is performed to copy it to the stack.
So I need to determine the best way to handle caching of stack entries. Perhaps instead of storing it in the local and stack treads they can be stored in the states themselves. It could essentially be a mapping of stack entries to locals for the most part. On entry of a new jump target, all cache entries will be invalidated for the most part. The thing to handle though would be the invalidation. I would then suppose that after each operation has been handled that it checks to see if the next operation or any jump targets would cause invalidation to occur. So the op parser will need to handle setting and handling of next jump targets.
So actually, I will need to handle cache flushing whenever there is code that would land on a jump target somewhere. Flushing must be performed before the jump code is generated. I was about to write it following it. So if in the event no flushing was performed, and it should be flushed then it will be flushed in the basic handler.
So yes, flushing if the next target is a jump target would be best and then handling flushing before a jump is performed (but after any conditions) would be best. Although it would not result in that optimized code, it should be good enough since most of the stack copies should be reduced.
Another optimization I will have to handle is dup and such. Say a value is read from a value such a dup is performed. For swap and other operations these would actually cause copies when stack entries are not sourced from locals. So as such, I will need to have caching on stack values too. Flushing would have to do both, although stack items are always temporary.
So next instruction to handle after a load is invoke of a special method. Special methods have a set of rules which change if the call acts just like a virtual call or it is an explicit method call. So as such, the linkage to use will vary. I suppose for simplicity if it maps to a virtual invoke that it will just the virtual invoke handler, while if it maps to a special one it will map to that. One thing to consider is that there will need to be multiple types of method references. So this means the global pool that is used will need method references for specials.
I will need to check if such a call is valid and determine that during link time. Since at JIT time it will not be known if a method can call another method. Using the basic pool information it will need call source (the current method and class) along with the target method and the invoke type it uses.
The generic pool of course can handle such things. I suppose to reduce the space complexity required to handle such things, I can have a reference to a method and then a call linkage associated with it. The call linkage has the from and to source and the type. So when a direct method reference is used it can point to the method data with its flags and such. Then the secondary linkage can use that information to determine if the call is valid or not. This way if a bunch of source methods end up calling a large number of methods there would be many duplicates of the source.