How can I use StringTokenizer instead of array?

Question

I'm currently redeveloping an OBJ & MTL importer for my game engine. At the moment, I'm focusing on increasing how quickly I can parse large OBJ files.

For now, I can parse about 1.2M vertices, uvs and normals in around 4-6 seconds. Which is awesome, considering the file that I'm parsing, couldn't be parsed originally.

Anyways, on with my question; I want to be able to make the following method use a StringTokenizer instead of String array. The reason is because using String.split is slow and I want to parse the file relatively fast.

private OBJIndex parseIndex(String token)
{   
    String[] tokens = token.split("/");

    OBJIndex result = new OBJIndex();
    result.setVertexIndex(Integer.parseInt(tokens[0]) - 1);

    if(tokens.length > 1)
    {
        if(!tokens[1].isEmpty())
        {
            hasUVs = true;
            result.setUvIndex(Integer.parseInt(tokens[1]) - 1);
        }

        if(tokens.length > 2)
        {
            hasNormals = true;
            result.setNormalIndex(Integer.parseInt(tokens[2]) - 1);
        }
    }

    return result;
}

How would I go about doing this?

What research have you done and where have you gotten stuck? — Phoenix, Jun 18 '17 at 16:11

Maik Semder · Accepted Answer · 2017-06-19 11:03:37Z

1

If you want fast loading times in your game, what engines do is: not using text based data-files (obj, xml ..) on big data like meshes in the first place.

Instead, prepare the data in an offline process, specifically for your engines needs and optimize this instead. In your engines data-exporter tool, read in the obj files and write the optimized data binary. Then there is no parser needed anymore when the game actually starts and runs, it just reads in the binary data directly into its data structures.

Unreal calls this process cooking, other engines have other terms for the same thing. The point is: optimize your engines data formats for fast loading, this excludes text based data formats for big data.

answered Jun 19 '17 at 11:03

Maik Semder

4,6461 gold badge23 silver badges24 bronze badges

1

\$\begingroup\$ The problem of parsing an obj file is still present here, it's just happening at import time in the editor rather than at runtime in the finished game. While our time requirements might be less exacting for offline baking processes, there are still good reasons to optimize this: it lets content creators iterate faster, and can save a lot of time whenever you need to do a big batch re-import because your binary format changed or somesuch. \$\endgroup\$ – DMGregory♦ Jun 19 '17 at 11:48
\$\begingroup\$ Agreed @DMGregory, cooking/baking times is not unimportant. However when it comes to optimization of game loading times on an engine level (that's how I understood the question), its best to go binary. That was my point, sorry if that wasn't clear from my post. \$\endgroup\$ – Maik Semder Jun 19 '17 at 12:55
\$\begingroup\$ Now that I read it again, I think you are right @DMGregory, it doesn't really answer the question as it was. Though it has the terms "for my game engine" in its question body, which changes the direction of the question a little and opens it for better optimization options. No harm done, Ill leave it here as an extended comment, maybe Mathew is interested in better optimizations, than the actual String.Split thing. \$\endgroup\$ – Maik Semder Jun 19 '17 at 13:14
\$\begingroup\$ Yep, you raise a perfectly valid point about data baking that's worth repeating. Implementing this suggestion efficiently will still need an answer to the OP's question about parsing the text data in the first place. \$\endgroup\$ – DMGregory♦ Jun 19 '17 at 13:17

add a comment |

Stack Exchange Network

current community

your communities

more stack exchange communities

How can I use StringTokenizer instead of array?

1 Answer 1

Your Answer

Hot Network Questions

How can I use StringTokenizer instead of array?

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Related

Hot Network Questions