Thursday, January 02, 2014

Per problem memory and Time limits in Greed

Greed (at least the git version) is now the first plugin to support the per problem memory and time limits that were recently added to topcoder.

I had to cry for help from the arena maintainers to get this to work. I discovered many things I didn't know about the arena API, for example, the Memory limit thing was in it since years ago (huh?). Anyway, I finally got it to work. That was quite a pull request...

Why did I care so much to get this to work? Well, if some TC problems are going to have a custom memory and time limit, you need to know. Currently I am reading problem statements generated by greed instead of those by the arena (it is much better!), so I need them to contain this information. There is also something else though...

My test system is basically just a runcpp.sh script that sets things up correctly and puts the default Memory Limit (256) and runs things. Then it calls the Greed-generated source code file, which has a time limit feature. So my current TopCoder environment makes quite some use of these limits. If a problem gets new limits, they would break without using this information.

Making the environment take advantage of this feature was another issue. For time limit, it was relatively easy, my tester templates already had a time limit constant. I just modified them to use a time factor instead of set limit. My templates are now an official part of greed and you can find more info at: https://github.com/shivawu/topcoder-greed/tree/master/src/main/resources/templates/dualcolor

The memory limit was an adventure of its own. Memory limit was enforced by the runcpp.sh (and similar) scripts I used. But now there was a custom ML per problem. I first needed to store that information somewhere. I am not sure if this was the simplest way possible, but what I did was add yet-another template to greed. The memory-limit template:

    memory-limit {
        overwrite = force
        outputFileName = "limit/${Problem.Name}.txt"
        templateFile = "limittemplate.txt"
    }

(You also need to add memory-limit to the templates line). This would make greed generate a file ProblemName.txt at a limit sub-folder of the contest folder... The template file is just this:

${Problem.MemoryLimitMB}

So this txt file just contains the memory limit in MB.

Next step was to make the runcpp.sh script able to read this txt file (if it exists). The same with the python and Java ones. Posting the Java one for reference as it is the easiest to understand (The other ones need to do far more complicated things like calling ulimits and multiplying the MB limit by 1024...):

#!/bin/bash
#default
LIMIT_MB=256

#extract folder name from $1:
DIR=$(dirname $1)
#extract file name (problem name) from $1:
PROBLEM=$(basename $1)
PROBLEM=${PROBLEM%%.*}
#build the limit file name
LIMIT=$DIR/limit/$PROBLEM.txt
if [ -f $LIMIT ]; then
#echo "READ MEMORY LIMIT"
LIMIT_MB=`cat $LIMIT`
#else
#echo "NO MEMORY LIMIT, USE DEFAULT"
fi
#echo "MB = " $LIMIT_MB

echo "compiling..."
javac $1.java
if [ $? = 0 ];
then
java -Xmx${LIMIT_MB}m $1
fi

The trick is to modify the java command line, to use -Xmx${LIMIT_MB}m, where LIMIT_MB contains the read memory limit value.

So?...

I just hope that problems with custom memory limits are not too rare to justify all this work.

I am liking greed more and more every day. Nowadays I consider Greed not to be a TopCoder arena plugin, but an arena plugin workshop. The workshop just happens to come with an example simple arena plugin by default :)

The arena plugin survey

Calling all Greed users (and any other plugin users too). It is best to get surveyed in the plugin survey at : http://apps.topcoder.com/forums/?module=Thread&threadID=807988&start=0 I already made a post for Greed, so if you use it, plus that post.

No comments :