How to ignore a “bad” character causing Alamofire to fail with response serialization?

Question

In 20 years of developing I always found all answers to my developing problems by searching online, and specifically on Stackoverflow. But this time I'm not able to find a solution for the sake of my life: I'm working on an iOS app which consumes a public api for searching timetables of public transport. Everything works fine, especially the Android version. But in iOS the serialization process fails if the response contains an invalid character.

I'm using Alamofire 5.2.1, Swift 5, Xcode 11.5 and the API is a DIVA/EFA service by Mentz, used mainly in central Europe (I think) for public transports.

The "bad" character which makes the serialization fail comes from an encoding issue I suppose, the JSON returned contains both the keys

"opPublicCode":"�BB"

and

"operator":"ÖBB"

(redundancy is another issue here, but nevermind). As you can see, this german Ö is once represented correctly, and once not. And it is this single character which then causes the following error generated by Alamofire:

responseSerializationFailed(
reason: Alamofire.AFError.ResponseSerializationFailureReason.decodingFailed
(error: Swift.DecodingError.dataCorrupted(Swift.DecodingError.Context(codingPath: [],
debugDescription: \"The given data was not valid JSON.\", 
underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 \"Unable to convert data to string around character 7100.\" 
UserInfo={NSDebugDescription=Unable to convert data to string around character 7100.})))))

This is the code for calling the api, nothing special I think:

AF.request(url)
    .responseDecodable(of: StaResponse.self) {(response: DataResponse<StaResponse, AFError> ) in
        switch response.result {
            case .success(let value):
                callback(value, nil)
            case .failure(let error):
                callback(nil, error)
            }
        }

and this the Codable object (I ignore the problematic value of "opPublicCode" by setting it to nil since I don't need it):

struct StaDiva : Codable {
    
    let branch : String?
    let dir : String?
    let line : String?
    let network : String?
    let opCode : String?
    let operatorName : String?
    let opPublicCode : String? = nil
    let project : String?
    let stateless : String?
    let supplement : String?
    let tripCode : String?
    
    enum CodingKeys: String, CodingKey {
        case branch = "branch"
        case dir = "dir"
        case line = "line"
        case network = "network"
        case opCode = "opCode"
        case operatorName = "operator"
        case project = "project"
        case stateless = "stateless"
        case supplement = "supplement"
        case tripCode = "tripCode"
    }
}

I already tried thousands of things, none of them worked. I set headers (Content-Encoding, Accept-Encoding, Accept), I tried with Alamofire 4, used.responseJSON and.responseString instead of.responseDecodable in my Alamofire request. I tried several options in my Codable struct, removing the "opPublicCode"-field, setting it to Data or Any instead of String. But with no luck. Every time the response contains results with this operator (of the trains, ÖBB), it fails.

What could I do here? Is there a way to ignore this character, maybe by replacing it? Did I miss something in the usage of Alamofire? Or is there a way to solve this in the Codable struct (but I don't think so, since the failure occurs before even trying do decoding something).

Needless to say, trying to have someone making some changes to the API is not an option.

If you need more info, I surely can give more details. Any help would be much appreciated! Thanks in advance!

Answer 1

Using Alamofire 5.2, you can use Alamofire's DataPreprocessor protocol to "fix" your payload before its parsed by responseDecodable . DataPreprocessor has a single requirement, func preprocess(_ data: Data) throws -> Data , which you can implement to fix your Data . So you'd implement something like this:

struct RepairPreprocessor: DataPreprocessor {
    func preprocess(_ data: Data) throws -> Data {
        // Find and remove or replace bad Data.
    }
}

You can then use it with responseDecodable :

.request(...)
.responseDecodable(of: SomeType.self, dataPreprocessor: RepairPreprocessor()) { response in 
    // Handle response.
}

This work is performed asynchronously, but given the apparent size of your response you'll need to cognizant of the performance impact this sort of Data parsing may have. I'd recommend operating on it as raw Data , with no String conversion, for better performance.

Additionally, you should report a bug to your backend vendor explaining that their responses have a bad encoding. It must be UTF-8 for it to be parsed properly.