Phoenix discussion

all the talk around the remake

Moderator: Deathifier

flowerpot
Assault Legionnaire
Posts: 74
Joined: Mon Dec 17, 2012 8:54 am
Location: Nowhere, trying to conjure up a Symbiot AI
Contact:

Phoenix discussion

Postby flowerpot » Tue Dec 06, 2016 2:22 am

A user on another forum asked about a place for general Phoenix discussion, so I decided to open such a thread here.
A concerned warlock gives you 3 technologies to even your odds against the other houses, and disappears into the wilderness. Of course, he may have a favor to ask you in the future...

mastnosis
Tank commander
Posts: 14
Joined: Fri Jun 16, 2017 2:12 am

Re: Phoenix discussion

Postby mastnosis » Sun Jun 18, 2017 10:19 am

Is there still any interest in this project. I've tried contacting the developers of both Phoenix and the EFS Editor direct via email with no responses.

I'm creating a JavaFX interface for the game. I've created a utility for reading in the Galaxy.gal file based on the source code, but there are definitely questions I have about some of the variables. I would prefer to have documentation that describes the byte format in detail. For instance why is the planet map info 44x65 instead of 44(width) x 32 (height). Especially the 65, at least 64 would be twice the height. In the source code it appears half the data is thrown away and the 65th row is discarded entirely. In the half that is thrown away, while most of it is zero some of it is not, so there must be some information encoded there that is being ignored.

Has anyone else viewed the source?

TKComposition
Infantery
Posts: 7
Joined: Thu Jun 01, 2017 4:34 am

Re: Phoenix discussion

Postby TKComposition » Mon Jun 19, 2017 5:35 am

I found Phoenix to be inspirational as well and have been slowly coding a clone and keeping a log of what I've learned so far. Although it would be great to have the developers insight into every file and variable, the answer may not be out there. It might be, but you can't necessarily count on it. Someone once encouraged me to keep pressing on to getting to something playable, even if you have to hack something up yourself for now. It could be leftovers from early development they just never took out. I'd encourage you to make a note and you can come back to it later if it's still important whenever later comes. Especially if your utility works for what you need it to do, it's probably not that important to understand right now. I haven't gotten to parsing the galaxy or reading the EFS Editor source, but if I get any insight when I do I'll share with the forum. Take care.

mastnosis
Tank commander
Posts: 14
Joined: Fri Jun 16, 2017 2:12 am

Re: Phoenix discussion

Postby mastnosis » Mon Jun 19, 2017 8:54 am

TKComposition wrote:I found Phoenix to be inspirational as well and have been slowly coding a clone and keeping a log of what I've learned so far... Someone once encouraged me to keep pressing on to getting to something playable,


Why are you choosing to start a new clone? Although Phoenix isn't complete, and I haven't played more than one or two turns of a new game with it, isn't it supposed to be 'playable'?

I have to admit myself, I'm quite conflicted on Phoenix. On the one hand I'm quite impressed with what it has accomplished so far, but it is clear by reading the code that the author is a self taught coder (which I applaud). The design violates practically every 'best practice' I know which makes it hard to jump into and contribute.

I'd really like to talk to someone familiar with the EFS Editor as I'm assuming it is more authoritative regarding the data files. I haven't spent much time reviewing the code since I don't have a decent C++ IDE installed and reading it in a simple text editor is tedious.

For now I'm just focused on creating a functional UI in JavaFx with no game logic attached. I can read the GALAXY.GAL file and load PCX and BIN image formats. I'll decide later if it is quicker to tack it on Phoenix or rewrite from scratch. I'm more likely to help with Phoenix if I can get hold of the developer but both email addresses I found bounced and no longer seem valid.

TKComposition
Infantery
Posts: 7
Joined: Thu Jun 01, 2017 4:34 am

Re: Phoenix discussion

Postby TKComposition » Tue Jun 20, 2017 2:28 am

I'm having fun making a clone and learning things about the game I never knew or appreciated before. Eventually I want to redesign the interface a bit, but I've got a long way to go right now. I'm not sure why your making another interface, but I hope it's fun for you and you're learning a lot. Enjoy the journey. Notepad++ might help you review the EFSEditor code. Take care.

mastnosis
Tank commander
Posts: 14
Joined: Fri Jun 16, 2017 2:12 am

Re: Phoenix discussion

Postby mastnosis » Tue Jun 20, 2017 3:58 am

I'm making the interface in JavaFX partly as a learning experience, but it also has many benefits including reduced CPU usage, easier customization via css, support for more platforms including tablets and phones. You will be able to resize the window to any arbitrary size.

mastnosis
Tank commander
Posts: 14
Joined: Fri Jun 16, 2017 2:12 am

Re: Phoenix discussion

Postby mastnosis » Mon Jun 26, 2017 9:13 am

In learning to parse the map data I discovered some funky behaviour which I can only attribute to indexing errors. I can imagine that this is causing some flakiness in the original game. The map data is organized in columns ( I would have done rows ), but the stranger aspect is that each column is 65 high instead of the expected 64 (double the number of map rows, one integer for resources and one for terrain info). This leads to an "extra" hex each two rows. I've double checked that this extra hex is off the map and cannot be seen. To me the only explanation is that someone used a "<=64" in an iterator in the original code when it should have been just "<". I can discern no other purpose to this invisible hex.
Another oddity related to the above is that resource information is stored in the following hex. What I mean by this is that if you see a "fertile" or "metal" icon, in a hex, the resource data is in the following hex RES:TERRAIN pair. I could imagine this to cause out of bounds exceptions if the above issue didn't exist. I could see the original programmers going to fix one bug but causing crashing in the game so they revert back to the original code as "it just works"

mastnosis
Tank commander
Posts: 14
Joined: Fri Jun 16, 2017 2:12 am

Re: Phoenix discussion

Postby mastnosis » Mon Jun 26, 2017 10:38 am

Ok, I fully decoded the planet terrain data. The format is 4bits terrain (16 possible 10 used), 5 bits graphics index (32 possible, number available depends on terrain). These can be stacked up to 3 deep. Top terrain is in the lowest bits, bottom terrain is in the high bits.
The programmer on Phoenix had to hand craft the map data rather than using the index which must have required a herculean effort.

Bottom Top
GRASS forest8 FOREST hill2 HILL
0001 01000 1000 00010 0111

flowerpot
Assault Legionnaire
Posts: 74
Joined: Mon Dec 17, 2012 8:54 am
Location: Nowhere, trying to conjure up a Symbiot AI
Contact:

Re: Phoenix discussion

Postby flowerpot » Tue Jun 27, 2017 4:57 am

mastnosis wrote:Is there still any interest in this project. I've tried contacting the developers of both Phoenix and the EFS Editor direct via email with no responses.


I don't recall receiving such email recently, but this would not be the first time the internet has eaten emails. When I collaborated with Richard Wein three years ago no emails he sent from his address arrived at gmail.com and I had to use another address ... also, I've been distracted IRL recently and am slowly returning to work on the project.

mastnosis wrote:I'm creating a JavaFX interface for the game. I've created a utility for reading in the Galaxy.gal file based on the source code, but there are definitely questions I have about some of the variables. I would prefer to have documentation that describes the byte format in detail. For instance why is the planet map info 44x65 instead of 44(width) x 32 (height). Especially the 65, at least 64 would be twice the height. In the source code it appears half the data is thrown away and the 65th row is discarded entirely. In the half that is thrown away, while most of it is zero some of it is not, so there must be some information encoded there that is being ignored.

Has anyone else viewed the source?


I would like to have documentation too, preferably in time for christmas 2012 ! Apart from the planet maps, the Galaxy.gal format is laid out in the EFS editor code. Regarding the planet maps, that 44x65 instead of 44x32 confused me a lot and it didn't help that there is/was a bug in the Phoenix code that processed the planet maps. It seems that the data is stored so that when viewed as 44x65 table it appears somewhat like in a hex map, every even numbered column (starting from zero) is offset downwards by one position like in the following picture where X is hex data and O unused/garbage bytes and 65th row is unused.

OXOX
XOXO
OXOX
XOXO

There is also some strangeness with the x,y positions of the units and cities, the y position ranges from 0 to 63 (instead of 0 to 31 as expected) with even rows having odd column numbers and odd rows having even column numbers (in Phoenix the y's are converted to 0-31.) It's been 3.5 years since I took a serious look at the planet maps on a bit-level, so the correctness of my memories may be in question ...

Don't waste your time wondering why something is done like its done ... after all these years, even the original EFS programmers do not know.

mastnosis wrote:Ok, I fully decoded the planet terrain data. The format is 4bits terrain (16 possible 10 used), 5 bits graphics index (32 possible, number available depends on terrain). These can be stacked up to 3 deep. Top terrain is in the lowest bits, bottom terrain is in the high bits.
The programmer on Phoenix had to hand craft the map data rather than using the index which must have required a herculean effort.


I used the map editor to create a test planet with all the possible forest hexes on plain grass, and then a bit of code to print out the index numbers for each different hex. Again, that was 3.5 years ago ...

mastnosis wrote:Another oddity related to the above is that resource information is stored in the following hex. What I mean by this is that if you see a "fertile" or "metal" icon, in a hex, the resource data is in the following hex RES:TERRAIN pair.


If I understood this correctly you are saying that you found some resource data in the seemingly mostly empty 44x65 planet map integers ? In the EFS editor code, which I used as a template for Phoenix galaxy.gal reading, the resources are read in among the structures. Internally, a harvestable resource is a city of type number 26-31 and the resources are treated and stored more or less like structures, which has it's own complications in original EFS.

In Phoenix the resources are initially read in among the other structures, but are then internally treated more separately than in the original EFS (in there air units can refuel on resource hexes and you can bombard (and destroy) resource hexes among other things, because the game/code treats them more or less like cities.) For Phoenix, I simply discarded half of the planet map data. It is possible that this is a remnant of some earlier version of EFS, or a half baked version of something that was never finished. There is plenty of this in EFS, a major source of confusion, and there may still be more.

mastnosis wrote:I have to admit myself, I'm quite conflicted on Phoenix. On the one hand I'm quite impressed with what it has accomplished so far, but it is clear by reading the code that the author is a self taught coder (which I applaud). The design violates practically every 'best practice' I know which makes it hard to jump into and contribute.


Actually I have a BSc in CS :) ... although I admit Phoenix code probably violates everything. When I started, at first, among other things, I considered doing the proper waterfall style analysis and design ... but after a few days I concluded that I would likely abandon the project without a single line of actual code written.

It's a huge project, I'm alone, I don't know if I have the necessary skills to do it (and at that point, one could argue I didn't yet have the skills, and I'm still not sure if I do now), I don't know if recreating EFS is possible with plain java, I don't know if contemporary processors can run the code fast enough. And then there is the Brooks maxim "always plan to throw one away; you will anyway." So best treat the whole project as a prototype ... and if it should work, fantastic!

If you haven't read it yet, this will give info on some of the choices made. Also, the Phoenix readme is a bit outdated but the contributing section does provide more info on choices made.

mastnosis wrote:For now I'm just focused on creating a functional UI in JavaFx with no game logic attached. I can read the GALAXY.GAL file and load PCX and BIN image formats. I'll decide later if it is quicker to tack it on Phoenix or rewrite from scratch. I'm more likely to help with Phoenix if I can get hold of the developer but both email addresses I found bounced and no longer seem valid.


Regarding Phoenix development, it's beginning to be somewhat playable ... except for the AI. Right now the AI is the main thing, without it the game is barely playable as a PBEM.
A concerned warlock gives you 3 technologies to even your odds against the other houses, and disappears into the wilderness. Of course, he may have a favor to ask you in the future...

mastnosis
Tank commander
Posts: 14
Joined: Fri Jun 16, 2017 2:12 am

Re: Phoenix discussion

Postby mastnosis » Tue Jun 27, 2017 11:22 pm

I don't recall receiving such email recently, but this would not be the first time the internet has eaten emails. When I collaborated with Richard Wein three years ago no emails he sent from his address arrived at gmail.com and I had to use another address ... also, I've been distracted IRL recently and am slowly returning to work on the project.

I don't think I tried a gmail account. I used what I could find in the source code and what is associated with the github account.
It seems that the data is stored so that when viewed as 44x65 table it appears somewhat like in a hex map, every even numbered column (starting from zero) is offset downwards by one position like in the following picture where X is hex data and O unused/garbage bytes and 65th row is unused.

OXOX
XOXO
OXOX
XOXO

There is also some strangeness with the x,y positions of the units and cities, the y position ranges from 0 to 63 (instead of 0 to 31 as expected) with even rows having odd column numbers and odd rows having even column numbers (in Phoenix the y's are converted to 0-31.) It's been 3.5 years since I took a serious look at the planet maps on a bit-level, so the correctness of my memories may be in question ...

Actually, the map data is stored as two integers per hex with the first integer representing the resources present and the second integer representing the terrain data. It actually is 32 hexes for all even columns and 33 hexes for all odd. I haven't decoded what the resource numbers represent yet, but I'm confident they represent resource info because the number align with resources visible on the map. They also seem to have something to do with resource producing factories. Phoenix basically ignores all the resource data and throws away the 33rd hex one every odd row.

I spent a couple of days pouring over the bits looking for patterns until it became clear to me. I wasn't satisfied with the decoding provided by Phoenix as although it looked 99% correct there were errors. Road hexes will obscure mountain for example. On reflection Phoenix is only decoding 2 possible terrain features when it is possible for hexes to include up to 4 (e.g. grass/forest/hill/road). Initially it threw me off that the second terrain data was shifted 9 bits. Seemed completely nonsensical not being on byte boundaries until I realized that the 5 bits were indexing which graphic to display. 4 bits are not enough for forest and water indexes, but 5 is. That is still not the whole picture. Rivers and Roads must use some game logic that handles orientation. Still, most of the heavy lifting is done via the index.

flowerpot, if you are interested in working together please dm me.

mastnosis
Tank commander
Posts: 14
Joined: Fri Jun 16, 2017 2:12 am

Re: Phoenix discussion

Postby mastnosis » Wed Jun 28, 2017 3:03 am

I did some more decoding of the map data. The river tiles need to be orientated based on their location. So for river tiles, of the 5 bits used for the index the first three bits are for choosing the correct tile and the 2 most significant bits are for orientation. Specifically the 8's bit high indicates flip graphic horizontally and the 16's bit means rotate 180 (rotated not flipped). Both bits high basically equals flip vertically.

mastnosis
Tank commander
Posts: 14
Joined: Fri Jun 16, 2017 2:12 am

Re: Phoenix discussion

Postby mastnosis » Wed Jun 28, 2017 3:20 am

Hold the presses! I realize the index data is variable width. Delta/Coastal tiles require rotation as well but they need all 5 bits for the index(22 tiles). But they also have the two additional bits for orientation increasing the control bits to 7 before beginning the next terrain feature in the chain. Before I was decoding 0B4A as tundra-0 delta-20 (0/101/10100/1010) when in fact it should be decoded as grass-0 delta-20-flip (0/1/0110100/1010)

I wonder if some of my other assumptions are wrong as well. Won't know until I have my display code fully up and running.

mastnosis
Tank commander
Posts: 14
Joined: Fri Jun 16, 2017 2:12 am

Re: Phoenix discussion

Postby mastnosis » Sun Jul 02, 2017 3:46 am

My planet map parsing is complete. I wanted to upload a screenshot, but it doesn't seem I can do that without first hosting it elsewhere.

I've done all I can with the available data. Some stuff has to be manually calculated like hex boundary bleed over and shorlines. This information will have to be calculated with respect to neighbours.

One of the peculiarities is that there seems to be a duplicate tile in the tilesets. The first two tiles are the same forest image. If there is a difference I cannot tell with my naked eyes and the forest do not match up unless you ignore the first. Have to skip over one in order for the forest tiles to match up properly.

TKComposition
Infantery
Posts: 7
Joined: Thu Jun 01, 2017 4:34 am

Re: Phoenix discussion

Postby TKComposition » Tue Jul 04, 2017 4:37 am

Just wanted to congratulate you on your milestone mastnosis. Keep up the enthusiasm.

Sukayo
Noble
Posts: 351
Joined: Thu Oct 12, 2000 1:01 am
Location: Brno, Czech Republic
Contact:

Re: Phoenix discussion

Postby Sukayo » Sat Jul 15, 2017 10:13 am

correct no uploads, for various reasons, some i forgot already possible.

But nice discussion going on - what i do not get is - when a clone, why you try to understand original data? Creating it new will at least when documented secure that you now every bit of info stored inside.


Return to “EFS Clone”

Who is online

Users browsing this forum: No registered users and 15 guests