Using sgn() To Clamp Values In Array Sorting Operations In ColdFusion

A couple of years ago, I ran into an issue with array.sort() in Lucee CFML wherein it threw an error if my sort operator returned a value that was larger than a signed integer. Yesterday, Adam Cameron left a comment on that post about using the sgn() function, in ColdFusion, to fix the issue. To be honest, I don’t believe I even knew that the sgn() function existed. So, in order to help fill in my lacking mental model, I wanted to quickly look at what the sgn() function does.

The sgn() function takes a numeric input and returns one of only three values:

• -1 : The input was less than 0.
• 0 : The input was 0 (or null in Lucee CFML).
• 1 : The input was greater than 0.

It just so happens that these are also the three documented, usable values for the .sort() functions in ColdFusion.

Now, let me quickly demonstrate the issue I was running into my previous post. Imagine that I have a list of database tables along with their current AUTO_INCREMENT primary key values. If I wanted to sort the the collection of tables based on their primary key, I could do something like this:

<cfscript>

	// NOTE: Max signed INT value in Java is 2147483647.
	tables = [
		{ name: "account", pkey: 64004394 },
		{ name: "company", pkey: 5307 },
		{ name: "file", pkey: 4947493641 },
		{ name: "file_folder", pkey: 3147493648 },
		{ name: "project", pkey: 839409903 },
		{ name: "share", pkey: 99347337683 },
		{ name: "user", pkey: 64004394 }
	];

	// Store tables by their primary key, descending.
	tables.sort(
		( a, b ) => {

			// If the primary keys in the two tables are the same, fall-back to using a
			// name-based sort in order to create a predictable outcome.
			if ( a.pkey == b.pkey ) {

				return( compareNoCase( a.name, b.name ) );

			}

			return( b.pkey - a.pkey );

		}
	);

	dump( tables );

</cfscript>

The max signed int in Java is 2,147,483,647. In Lucee CFML, the .sort() operator return value must fit into a signed int. However, because my primary key values are so large, in some cases, the maths in my return statement:

return( b.pkey - a.pkey );

… will generate a value that is larger than a signed int. And, running this in Lucee CFML 5.3.8.201 will throw the following error:

lucee.runtime.exp.FunctionException: invalid call of the function ArraySort, second Argument (function) is invalid, return value of the function [lambda_gj] cannot be casted to an integer.

To fix this problem, as Adam Cameron pointed out, we can simply wrap our return value in sgn() to clamp the possible return values to -1, 0, and 1:

<cfscript>

	// NOTE: Max signed INT value in Java is 2147483647.
	tables = [
		{ name: "account", pkey: 64004394 },
		{ name: "company", pkey: 5307 },
		{ name: "file", pkey: 4947493641 },
		{ name: "file_folder", pkey: 3147493648 },
		{ name: "project", pkey: 839409903 },
		{ name: "share", pkey: 99347337683 },
		{ name: "user", pkey: 64004394 }
	];

	// Store tables by their primary key, descending.
	tables.sort(
		( a, b ) => {

			// If the primary keys in the two tables are the same, fall-back to using a
			// name-based sort in order to create a predictable outcome.
			if ( a.pkey == b.pkey ) {

				return( compareNoCase( a.name, b.name ) );

			}

			// The sgn() function will only ever return -1, 0, 1 for a given input. As
			// such, we can clamp our delta down to a value that will always work for the
			// sort operation.
			return( sgn( b.pkey - a.pkey ) );

		}
	);

	dump( tables );

</cfscript>

Now, when we run this ColdFusion code in Lucee CFML 5.3.8.201, we get the following output:

As you can see, with our sgn() safety-net in place, our .sort() operator runs perfectly well.

Huge shout out to Adam Cameron for bringing the sgn() function to my attention. It seems like it provides a perfect intent for the sort functions in ColdFusion. I can’t immediately think of any other reason to use this function. But, now that I know that it exists, I’ll be on the look-out.