Shortest path between two nodes with fixed number of nodes in path

Question

I have a weighted graph with around 800 nodes, each with a number of connections ranging from 1 to around 300. I need to find the shortest (lowest cost) path between two nodes with some extra criteria:

• The path must contain exactly five nodes.
• Each node has an attribute (called position in the example code) that takes one of five values; the five nodes in the path must all have unique values for this attribute.
• The algorithm needs to allow for 1-2 required nodes to be specified that the path must contain at some point in any order.
• The algorithm needs to take less than 10 seconds to run, preferably as little time as possible while losing as little accuracy as possible.

My current solution in Python is to run a Depth-Limited Depth-First Search which recursively searches every possible path. To make this algorithm run in reasonable time I have introduced a limit to the number of neighbour nodes that are searched at each recursion level. This number can be lowered to decrease the computation time but at the cost of accuracy. Currently this algorithm is far too slow, with my most recent test coming in at 75 seconds with a neighbour limit of 30. If I decrease this neighbour limit any more, my testing shows that the accuracy of the algorithm begins to suffer badly. I am out of ideas on how to solve this problem while satisfying all of the above criteria. My code is as follows:

# The path must go from start -> end, be of length 5 and contain all nodes in middle
# Each node is represented as a tuple: (value, position)
def dfs(start, end, middle=[], path=Path(), best=Path([], math.inf)):
    # If this is the first level of recursion, initialise the path variable
    if len(path) == 0:
        path = Path([start])
    
    # If the max depth has been exceeded, check if the current node is the goal node
    if len(path) >= depth:
        # If it is, save the path
        # Check that all required nodes have been visited
        if len(path) == depth and start == end and path.cost < best.cost and all(x in path.path for x in middle):
            # Store the new best path
            best.path = path.path
            best.cost = path.cost
        return
    
    # Run DFS on all of the neighbors of the node that haven't been searched already
    # Use the weights of the neighbors as a heuristic; sort by lowest weight first
    neighbors = sorted([x for x in graph.get(*start).connected_nodes], key=lambda x: graph.weight(start, x))
    # Make sure that all neighbors haven't been visited yet and that their positions aren't already accounted for
    positions = [x[1] for x in path.path]
    # Only visit neighbouring nodes with unique positions and ids
    filtered = [x for x in neighbors if x not in path.path and x[1] not in positions]
    for neighbor in filtered[:neighbor_limit]:
        if neighbor not in path.path:
            dfs(neighbor, end, middle, Path(path.path + [neighbor], path.cost + graph.weight(start, neighbor)), best)
    
    return best

Path Class:

class Path:
    def __init__(self, path=[], cost=0):
        self.path = path
        self.cost = cost
    
    def __len__(self):
        return len(self.path)

Any help in improving this algorithm or even suggestions on a better approach to the problem would be much appreciated, thanks in advance!

Answer 1

You should iterate over all possible orderings of the 'position' attribute, and for each one use Dijkstra's algorithm or BFS to find the shortest path that respects that ordering.

Since you know the position of the first and last nodes, there are only 3! = 6 different orderings for the intermediate nodes, so you only have to run Dijkstra's algorithm 6 times.

Even in python, this shouldn't take more than a couple hundred milliseconds to run, based on the input sizes you provided.