Some interesting results after heavy tweaking

a place to discuss it

Moderators: Deathifier, Sukayo

User avatar
Eightball Maniac
Assault Legionnaire
Posts: 94
Joined: Fri Aug 11, 2006 1:01 am
Location: Holding the line

Postby Eightball Maniac » Wed May 13, 2015 2:37 am

Been a long time but I'm finally getting back into EFS. Played around with the last version of this mod I worked on to see where I left off and already found stuff that needed changing. Ah, modding.

flowerpot
Assault Legionnaire
Posts: 74
Joined: Mon Dec 17, 2012 8:54 am
Location: Nowhere, trying to conjure up a Symbiot AI
Contact:

Re:

Postby flowerpot » Wed Dec 09, 2015 7:48 pm

Eightball Maniac wrote:Been a long time but I'm finally getting back into EFS. Played around with the last version of this mod I worked on to see where I left off and already found stuff that needed changing. Ah, modding.


I downloaded the mod in the first post, and got three errors with Phoenix. Two are Phoenix bugs, but there is possibly an error in UNIT.DAT on line 293 with "Direwolf Assault Tank", the req'd tech is 117 0 0 0 when it should be perhaps 114 0 0 0 ?
A concerned warlock gives you 3 technologies to even your odds against the other houses, and disappears into the wilderness. Of course, he may have a favor to ask you in the future...

User avatar
Eightball Maniac
Assault Legionnaire
Posts: 94
Joined: Fri Aug 11, 2006 1:01 am
Location: Holding the line

Re: Some interesting results after heavy tweaking

Postby Eightball Maniac » Wed Dec 09, 2015 10:03 pm

The last release was from so long ago I can't remember what was and wasn't changed in the time since. In the current version the Direwolf relies on tech 115, but techs end at 113 after some cleanup. Good catch! Does Phoenix have some kind of log file that lists errors? That would be most useful. There are probably a vast number of similar bugs lurking in the mod files.

flowerpot
Assault Legionnaire
Posts: 74
Joined: Mon Dec 17, 2012 8:54 am
Location: Nowhere, trying to conjure up a Symbiot AI
Contact:

Re: Some interesting results after heavy tweaking

Postby flowerpot » Wed Dec 09, 2015 11:28 pm

Eightball Maniac wrote:The last release was from so long ago I can't remember what was and wasn't changed in the time since. In the current version the Direwolf relies on tech 115, but techs end at 113 after some cleanup. Good catch! Does Phoenix have some kind of log file that lists errors? That would be most useful. There are probably a vast number of similar bugs lurking in the mod files.


Yes, in "phoenixlog.txt" and five previous logs in "phoenixlog.txt.1-5". The errors are also shown on console if you started on commandline.

Below are the two logs concatenated, the first was caused by a Phoenix bug unable to process the DAT file line 36. From the second you can see (well knowing some java and having coded the stuff I can :) ) that some unit reports a needed tech with index greater than the number of techs in the Phoenix tech array. Of course, it does not immediately tell who is wrong, and I first suspected a Phoenix bug, because the UNIT.DAT format looked ok, but then searched for "117" and Direwolf was the only hit.

Code: Select all

***** Begin Log Entry Wed Dec 09 00:05:37 EET 2015 *****
Error reading file: DAT\STRBUILD.DAT, line: 36

***** Begin Stack Trace Wed Dec 09 19:02:36 EET 2015 *****
Thread[AWT-EventQueue-0,6,main]
java.lang.ArrayIndexOutOfBoundsException: 117
        at game.Research.checkNeededTechs(Research.java:162)
        at game.Research.setCanBuild(Research.java:138)
        at game.Research.<init>(Research.java:72)
        at game.Faction.initResearch(Faction.java:99)
        at game.Faction.<init>(Faction.java:90)
        at game.Game.init(Game.java:229)
        at gui.Gui.<init>(Gui.java:264)
        at gui.Gui.createAndShowGUI(Gui.java:2418)
        at gui.Gui.access$2600(Gui.java:109)
        at gui.Gui$59.run(Gui.java:2450)
        at java.awt.event.InvocationEvent.dispatch(Unknown Source)
        at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
        at java.awt.EventQueue.access$500(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue.dispatchEvent(Unknown Source)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.run(Unknown Source)
***** End Stack Trace Wed Dec 09 19:02:36 EET 2015 *****


The first bug is in 0.50 so I may have to release 0.50.1 ... but it's gonna take at least an hour for the automated tests to run after the fix is in.

(There was also a third error which is in the prototype Symbiot AI but this is for 0.51)

Also, at program start Phoenix prints out a list of all stacks with more than 20 units. Your mod reports (I've seen this in Hyperion and Emperor Wars also, so a bug in the GALAXY maker tools ?):

Code: Select all

Stack size 21 on CHERNOBOG at (28,13) faction Stigmata
Stack size 22 on STIGMATA at (18,16) faction Stigmata
Stack size 24 on STIGMATA at (30,25) faction Stigmata
Stack size 21 on NOWHERE at (41,8) faction Eye
Stack size 24 on CRITICORUM at (27,10) faction Rebels
Stack size 21 on ISTAKHR at (2,20) faction Rebels
Stack size 22 on ISTAKHR at (15,14) faction Rebels
Stack size 22 on ISTAKHR at (26,27) faction Al Malik
Stack size 22 on ISTAKHR at (39,24) faction Church
Stack size 26 on ARAGON at (43,6) faction Hazat
Stack size 23 on LEAGUEHEIM at (1,15) faction League




This provokes the question: "What does all this do to EFS1.4 EFS.EXE games ? Sudden crash/AI endless loop when someone tries to build a Direwolf/move an oversize stack ? No requisite for Direwolves ? Direwolves impossible to build ?"
A concerned warlock gives you 3 technologies to even your odds against the other houses, and disappears into the wilderness. Of course, he may have a favor to ask you in the future...

User avatar
Eightball Maniac
Assault Legionnaire
Posts: 94
Joined: Fri Aug 11, 2006 1:01 am
Location: Holding the line

Re: Some interesting results after heavy tweaking

Postby Eightball Maniac » Thu Dec 10, 2015 1:35 am

flowerpot wrote:Yes, in "phoenixlog.txt" and five previous logs in "phoenixlog.txt.1-5". The errors are also shown on console if you started on commandline.

Below are the two logs concatenated, the first was caused by a Phoenix bug unable to process the DAT file line 36. From the second you can see (well knowing some java and having coded the stuff I can :) ) that some unit reports a needed tech with index greater than the number of techs in the Phoenix tech array. Of course, it does not immediately tell who is wrong, and I first suspected a Phoenix bug, because the UNIT.DAT format looked ok, but then searched for "117" and Direwolf was the only hit.


Very useful! Games that have error logs are so much easier to mod for when it gives you even a clue as to what causes a crash or similar. I'll have to start testing with Phoenix to clear up problems.

flowerpot wrote:Also, at program start Phoenix prints out a list of all stacks with more than 20 units. Your mod reports (I've seen this in Hyperion and Emperor Wars also, so a bug in the GALAXY maker tools ?):


The map editor doesn't give you any warning that there may be more than the maximum of 20 units in a stack. Units can be placed either individually or in groups with a slider. Short of carefully counting out the units in every stack you place (a huge task when you're populating 40 planets), mistakes are inevitable. :(

flowerpot wrote:This provokes the question: "What does all this do to EFS1.4 EFS.EXE games ? Sudden crash/AI endless loop when someone tries to build a Direwolf/move an oversize stack ? No requisite for Direwolves ? Direwolves impossible to build ?"


Direwolves simply aren't buildable and don't appear in the unit list. Any Direwolves placed via the map editor function normally. As for oversized stacks, I think the excess units appear once the others have been moved? It doesn't cause crashes in my experience.

flowerpot
Assault Legionnaire
Posts: 74
Joined: Mon Dec 17, 2012 8:54 am
Location: Nowhere, trying to conjure up a Symbiot AI
Contact:

Re: Some interesting results after heavy tweaking

Postby flowerpot » Sat Dec 12, 2015 2:14 am

Eightball Maniac wrote:
flowerpot wrote:Yes, in "phoenixlog.txt" and five previous logs in "phoenixlog.txt.1-5". The errors are also shown on console if you started on commandline.


Very useful! Games that have error logs are so much easier to mod for when it gives you even a clue as to what causes a crash or similar. I'll have to start testing with Phoenix to clear up problems.


Hrm :oops: ... 0.50 seems to be the first release since I implemented high level automatic testing. All that stuff was a first for me, and among other things I had not put the test data into version management, so presently the test data is in 0.51 development state. So, to properly release a 0.50.1 bugfix, I would have to do some backtracking with the tests. I'd rather spend that time trying to transform lead into gold with the Symbiot AI for the 0.51 release.

Eightball Maniac wrote:
flowerpot wrote:Also, at program start Phoenix prints out a list of all stacks with more than 20 units. Your mod reports (I've seen this in Hyperion and Emperor Wars also, so a bug in the GALAXY maker tools ?):


The map editor doesn't give you any warning that there may be more than the maximum of 20 units in a stack. Units can be placed either individually or in groups with a slider. Short of carefully counting out the units in every stack you place (a huge task when you're populating 40 planets), mistakes are inevitable. :(


Well, in each GALAXY.GAL I have only seen a dozen of those, they should be quickly fixable ... updated modding tools might be a possible future project. In fact, that does not necessarily require much more than writing code to save galaxies in EFS1.4 GALAXY.GAL format. For testing purposes, I already have a crude wiz-mode tool for inserting basic units.
A concerned warlock gives you 3 technologies to even your odds against the other houses, and disappears into the wilderness. Of course, he may have a favor to ask you in the future...

User avatar
Eightball Maniac
Assault Legionnaire
Posts: 94
Joined: Fri Aug 11, 2006 1:01 am
Location: Holding the line

Re: Some interesting results after heavy tweaking

Postby Eightball Maniac » Mon Dec 14, 2015 6:11 am

flowerpot wrote:Hrm :oops: ... 0.50 seems to be the first release since I implemented high level automatic testing. All that stuff was a first for me, and among other things I had not put the test data into version management, so presently the test data is in 0.51 development state. So, to properly release a 0.50.1 bugfix, I would have to do some backtracking with the tests. I'd rather spend that time trying to transform lead into gold with the Symbiot AI for the 0.51 release.


Indeed, it can wait for 0.51.

flowerpot wrote:Well, in each GALAXY.GAL I have only seen a dozen of those, they should be quickly fixable ... updated modding tools might be a possible future project. In fact, that does not necessarily require much more than writing code to save galaxies in EFS1.4 GALAXY.GAL format. For testing purposes, I already have a crude wiz-mode tool for inserting basic units.


I'm doing some planet overhauls so I'll see about fixing those oversized stacks while I'm at it.

An updated and expanded map editor would be fantastic eventually. There were various attempts over the years, but none I've seen were even close to being as effective as the official one. There are a lot of things I'd like to see changed or improved in both game and modding aspects as well, but I'll avoid listing them. You've got plenty on your plate as it is!

flowerpot
Assault Legionnaire
Posts: 74
Joined: Mon Dec 17, 2012 8:54 am
Location: Nowhere, trying to conjure up a Symbiot AI
Contact:

Re: Some interesting results after heavy tweaking

Postby flowerpot » Sun Dec 27, 2015 2:08 am

Update on the Symbiot AI:

Symbiots just conquered Absolution and Daishan, transporting ground troops as necessary. After that they saw no enemy cities to conquer on the mapped planets (Chernobog, Absolution and Daishan) so they sent scouts to Stigmata space. This happened on A.D. 4968. From Stigmata orbit they saw potential new enemy cities to conquer so they transported ground troops to Stigmata (and used a space carrier captured from the formerly rebel held church on Daishan to complement their pod ships.) The static Stigmata city defences were overrun by A.D. 4989. Next, scouts were sent to Nowhere, Bannockburn, Istakhr, and Shaprut. Ferrying of ground troops to Nowhere commenced but by A.D. 5007 the generation 1 AI ran out of troops, since it does not build units (or cities) yet (also PTS needs to be adapted for AI use since it happens outside of normal Phoenix stack movement ... and air units are not utilized yet ... Symbiots do not seem to have their own naval units, but they might capture some ...) By A.D. 5056 almost all house assets on ByzII had died of hunger, but back at the home planets they had harvested some serious resources :)

In other news:

A bug created long ago outside the AI showed up on the second test run. Routing was implemented in version 6 (that would have been v0.6.0 in the currently used semantic versioning), before city capture and lots of other things, and has not been updated since. Apparently, running random AI test games might be a good way of finding some esoteric bugs in the core game logic. So the bug is :oops: but finding it like this is :) ...

so this bug needs fixing ASAP, and then AI PTS and unit/city building.
A concerned warlock gives you 3 technologies to even your odds against the other houses, and disappears into the wilderness. Of course, he may have a favor to ask you in the future...

User avatar
Eightball Maniac
Assault Legionnaire
Posts: 94
Joined: Fri Aug 11, 2006 1:01 am
Location: Holding the line

Re: Some interesting results after heavy tweaking

Postby Eightball Maniac » Mon Dec 28, 2015 10:44 pm

I do enjoy these descriptions of what happens during AI testing. Most promising. The stock AI for Symbiots was pretty good in EFS, at least once Hyperion figured out how to let them produce new units. They happily invade anything they can and will try to reinforce any Symbiot units that crop up on other planets from ruins. All the more reason for players to join in the defense of Stigmata!

flowerpot
Assault Legionnaire
Posts: 74
Joined: Mon Dec 17, 2012 8:54 am
Location: Nowhere, trying to conjure up a Symbiot AI
Contact:

Re: Some interesting results after heavy tweaking

Postby flowerpot » Tue Dec 29, 2015 10:33 pm

Eightball Maniac wrote:I do enjoy these descriptions of what happens during AI testing. Most promising. The stock AI for Symbiots was pretty good in EFS, at least once Hyperion figured out how to let them produce new units. They happily invade anything they can and will try to reinforce any Symbiot units that crop up on other planets from ruins. All the more reason for players to join in the defense of Stigmata!


Well, apparently the bug was an easy fix and adding PTS to AI was not too difficult either.

Now there is some balancing ahead: in vanilla 1.4 with the initial forces available to the Symbiots, they can completely destroy the beginning Fleet & Stigmata space stacks and PTS defences in three years. After that, annihilation of all opposition on Stigmata surface could be done before the first regent elections (in 10 years), I think even if both the Fleet and Stigmata were in capable hands (human or computer) from the start. It's a no contest if the Symbiots concentrate their firepower. Additionally, Symbiot units do not require resources, so the most powerful units can be spammed at will (if they start by spamming nesters, they will have all their planets full of stacks soon enough.)

So, some things to consider:
  1. initial symbiot forces may be enough to overrun initial Stigmata defences before players can do anything
  2. symbiots can spam new units
  3. with space spotting fixed, or "fixed" as we do not really know what the intent with (space) spotting was, all symbiot space units have more camo than opposing units have spot and thus are invisible unless they attack. So they can sneak past Stigmata at will, there really is no blockade.
How to handle these:
  1. limiting aggression, based on stigmata defence capability, seems the only choice here
  2. some sort of building and/or maintenance cost for symbiot units could be calculated based on unit stats and some sort of allowance could be calculated based on how many cities they have, also some minimum hex distance between any two hives could be defined
    1. make space units like tracker legions are for foot units, this is a modding matter and (I think) out of context for Phoenix.
    2. program some variation into spotting, eg. a spotting "to hit" roll which adds some random value to actual spotting and/or larger fleets equals less chance of hiding (this would be sort of included with the "to hit" roll if spotting was rolled separately for each ship since chances of spotting at least one go up exponentially as stack size increases) ... this could be made a modding matter by introducing an optional SPOTROLL.DAT in PHOENIX dir which could contain some configurable spotting variables/randomization

I reckon these are sort of positive obstacles, in the sense that the first assumption is that making a an AI better than a pushover is always very hard ... of course, successful balancing may turn out to be the real challenge.

Also, looking at the above, Phoenix might be at a stage where actual play testing by real players becomes increasingly necessary ... :shock:
A concerned warlock gives you 3 technologies to even your odds against the other houses, and disappears into the wilderness. Of course, he may have a favor to ask you in the future...

User avatar
Eightball Maniac
Assault Legionnaire
Posts: 94
Joined: Fri Aug 11, 2006 1:01 am
Location: Holding the line

Re: Some interesting results after heavy tweaking

Postby Eightball Maniac » Sun Jan 03, 2016 8:12 am

flowerpot wrote:How to handle these:
  1. limiting aggression, based on stigmata defence capability, seems the only choice here
  2. some sort of building and/or maintenance cost for symbiot units could be calculated based on unit stats and some sort of allowance could be calculated based on how many cities they have, also some minimum hex distance between any two hives could be defined
    1. make space units like tracker legions are for foot units, this is a modding matter and (I think) out of context for Phoenix.
    2. program some variation into spotting, eg. a spotting "to hit" roll which adds some random value to actual spotting and/or larger fleets equals less chance of hiding (this would be sort of included with the "to hit" roll if spotting was rolled separately for each ship since chances of spotting at least one go up exponentially as stack size increases) ... this could be made a modding matter by introducing an optional SPOTROLL.DAT in PHOENIX dir which could contain some configurable spotting variables/randomization


  1. I'd opt for a limit on the number of Symbiot units before an aggression limit, perhaps configurable through a dat or ini file. There could also be a warmup period where the Symbiots send in smaller probing attacks before they begin attacking in full force, giving players a chance to send units over to help the Garrison. Perhaps tied to the mean tech level of Houses?
  2. Their better units can be slowed down with increased build times. Hives could indeed have a minimum distance between them, as with mines/farms/wells.
    1. It could indeed be left to modding. Perhaps the camo on Symbiot ships is reduced for a vanilla game or certain House ships have their spotting ability increased.
    2. I'd like to see this implemented if you're willing. Huge fleets are easy to spot unless you have no ships in orbit at all, but smaller ships running alone could slip past a fleet without some dedicated spotters.

flowerpot
Assault Legionnaire
Posts: 74
Joined: Mon Dec 17, 2012 8:54 am
Location: Nowhere, trying to conjure up a Symbiot AI
Contact:

Re: Some interesting results after heavy tweaking

Postby flowerpot » Fri Jan 29, 2016 12:29 am

Update on the Symbiot AI:

New Units are now built in existing hives. However, the first test game saw Stigmata conquered in about 110 years. This is actually expected since the various parts of the AI -- build, scout, transport and conquer -- are each implemented with minimal effort and complexity, and thus do not operate well alone, much less together.

An automated AI testing rig is now under development. Currently, it runs the symbiot AI against static opponents, until all cities on Stigmata have been conquered. The first five games stopped at the following years:
***** year 5016 *****
***** year 5029 *****
***** year 5047 *****
***** year 5071 *****
***** year 5036 *****

So shortest time to conquer Absolution, Daishan and Stigmata was 60 years and longest was 115 years. That's quite a deviation which suggests that in general we are dealing with a very low quality AI since it varies so much depending only on what the initial seed for the rng is. Most of this probably has to do with the AI not knowing how to deal with PTS, and it hurts a lot. So the next things to do is to make the AI concentrate forces more, land beyond known PTS defences and/or bombard known PTS defences.
A concerned warlock gives you 3 technologies to even your odds against the other houses, and disappears into the wilderness. Of course, he may have a favor to ask you in the future...

Bfriudo
Fighter pilot
Posts: 22
Joined: Wed Mar 02, 2011 11:18 pm
Location: Italy

Re: Some interesting results after heavy tweaking

Postby Bfriudo » Mon Feb 15, 2016 8:29 pm

Oy! Been so long, checking in for no apparent reason and lo, there's fresh news on the line! This game still got a hold on us it seems. Good to read from you all once again and I hope everything's fine. Cheers and keep up with the modding! .)

beetlecat
Infantery
Posts: 9
Joined: Wed Jul 14, 2004 1:01 am
Location: Seattle, WA

Re: Some interesting results after heavy tweaking

Postby beetlecat » Tue Feb 23, 2016 11:46 pm

Bfriudo wrote:Oy! Been so long, checking in for no apparent reason and lo, there's fresh news on the line! This game still got a hold on us it seems. Good to read from you all once again and I hope everything's fine. Cheers and keep up with the modding! .)


Exactly. I just saw my game disc over the weekend, and kept thinking about it until I did some searching, and hit on the new Phoenix efforts, as well as re-downloading Hyperion and Nova -- and now Emperor Wars?

Looking forward to the AI fixes :)

User avatar
Eightball Maniac
Assault Legionnaire
Posts: 94
Joined: Fri Aug 11, 2006 1:01 am
Location: Holding the line

Re: Some interesting results after heavy tweaking

Postby Eightball Maniac » Thu Feb 22, 2018 11:01 pm

Been poking at this again to see where I left off last time.

I had spent some time cleaning up the mod, removing extraneous new units and overhauling the starting units each faction has, so it's not as wacky as it used to be with tremendous numbers of high tech units for rebels and the player factions. A few high tech units still exist, but it's mostly early game units now.

Early game is fairly difficult, since the player needs to increase food production and drastically reduce costs ASAP. Slamming units into rebels to whittle down your own forces is about the best way to do it, along with capturing more cities to increase tax income and farms to feed ravenous citizens. I'll need to test each faction to make sure they're of roughly similar difficulty.

Engineers using hover movement may not be the best, since the AI loves constructing roads across lakes and oceans to build in other areas which can lead to oddities. Players cannot build roads across water, so it's purely an AI advantage if you want to call it that. Unexpected AI access routes are tempting, but I fear long term it would just make a mess. I'll leave it in for now to see what happens late game. Might be an interesting alternative to sea transports, which the AI cannot use.

I rebuilt several planet maps based on the maps in the Fading Suns RPG books, but some are a bit plain. Not sure if I had finished or was still working on them. Either way, will need to add some extra details to make them a little more interesting. Terrain aside, multiple factions exist on most planets now, be it homeworlds, colonies, or random nearby worlds. This offers additional possibilities for conflict between factions, and perhaps some incentive to go to war early.


Return to “Fixing it ourself”

Who is online

Users browsing this forum: No registered users and 6 guests