Rivian says “fat finger” caused software update to brick infotainment systems, physical servicing may be required::Today’s cars are more like computers on wheels, and even a seemingly routine software update can lead to unexpected consequences. Rivian unfortunately experienced a “fat finger” mishap with their latest software update, bricking infotainment systems […]

  • Destraight@lemm.ee
    link
    fedilink
    English
    arrow-up
    82
    arrow-down
    9
    ·
    1 year ago

    I don’t have to update my car in order to drive it. Sucks to be them

    • LilB0kChoy@lemm.ee
      link
      fedilink
      English
      arrow-up
      21
      arrow-down
      2
      ·
      1 year ago

      I thought the Rivian vehicles were still drivable? It was just the infotainment that’s impacted, no?

        • LilB0kChoy@lemm.ee
          link
          fedilink
          English
          arrow-up
          18
          arrow-down
          6
          ·
          1 year ago

          That’s always going to be a problem for early adopters of new tech. Plus, since it’s automotive you’d run that risk with the first model year or two after a major redesign too.

          If what Rivian has reported is true it’s not really a failing of the equipment but human error. That’s always going to be tough to account for, though there’s likely improvements to be made to their testing methodology before pushing to prod.

      • Destraight@lemm.ee
        link
        fedilink
        English
        arrow-up
        18
        arrow-down
        18
        ·
        1 year ago

        Ok man what if it’s an emergency. Here’s an example: My wife is in labour, but my car has to update so I can’t drive her to the hospital. Now I’ll have to pay a more than 2 grand ambulance ride since it was out of network.

        • shastaxc@lemm.ee
          link
          fedilink
          English
          arrow-up
          28
          arrow-down
          2
          ·
          1 year ago

          Fortunately, you can still do that with a Rivian. From the article that no one read:

          Speedometer, charging, backup cameras, locks, lights, wipers, and turn signals are all still functional with the 2023.42 error.

          • LCP@lemmy.world
            link
            fedilink
            English
            arrow-up
            8
            ·
            edit-2
            1 year ago

            At least there’s a speedometer on a separate display.

            On the Tesla Model Y there’s only one centre display for everything. I was on a road trip with someone in one and the display froze then automatically rebooted itself. Took a good 5 minutes.

            Nothing happened thankfully and the car kept running, but it’s a scary experience when you’re driving 65-70 MPH on a highway. I was worried the car would refuse to continue driving and lock itself or something while it rebooted.

          • originalucifer@moist.catsweat.com
            link
            fedilink
            arrow-up
            15
            arrow-down
            3
            ·
            1 year ago

            hahah nothing says america like absolutely obscene healthcare costs… cept maybe all the human killing devices they love so very very much

        • mack7400@lemmy.world
          link
          fedilink
          English
          arrow-up
          5
          arrow-down
          1
          ·
          1 year ago

          Considering I have a decent chance of getting a “Please wait – updating” message when trying to play a damn videogame nowadays, that’s not out of the realm of possibility.

        • gimmemahlulz@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          8
          arrow-down
          5
          ·
          1 year ago

          At least with a tesla you have to manually confirm the update, and can cancel out of it for 2 minutes after doing so. If something happens between then and 15 minutes later then i guess you’re fucked, but with the amount of control you have this really should never be a problem.

        • Imgonnatrythis@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          7
          arrow-down
          6
          ·
          1 year ago

          You are inconsistent. If you are going to adopt a luddite way of life, have her deliver at home not at the fancy high tech hospital where the computerized anesthesia machine might need to update before the c section incision is made.

        • LUHG@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          5
          ·
          1 year ago

          If you were going to drive you’d get a taxi instead. I agree but just saying.

  • neanderthal@lemmy.world
    link
    fedilink
    English
    arrow-up
    52
    arrow-down
    1
    ·
    1 year ago

    I would think they would keep at least 1 of each model/trim of vehicle for testing these things. This leads me to believe one of the following:

    • Too tight of deadlines
    • Cheap management won’t pay for testing time or units
    • Culture of pencil whipping
    • A bad apple didn’t do their job, which should be caught by procedures
      • DigitalTraveler42@lemmy.world
        link
        fedilink
        English
        arrow-up
        13
        ·
        edit-2
        1 year ago

        That’s typical of the automotive industry, they don’t pay anyone reasonably, they treat the workers like shit, and then they get mad when the workers protest, unionize, or quit. This is how the automotive industry works from the top down, from the highest executive to the lowliest salesman.

        Also from an IT/Development perspective, the money is usually meh, but also the technology is usually expected to be cutting edge, while on the backend they’re cutting corners and costs, thus undermining that “cutting edge” factor.

    • grue@lemmy.world
      link
      fedilink
      English
      arrow-up
      11
      ·
      1 year ago

      I think it’s more likely just typical dysfunctional software industry workflows. Companies that actually test their software adequately before deployment are the exception, not the norm. That’s different from what you said in your second bullet point because it’s not even an issue of cheapness, it’s an issue of not actually understanding what the best practices need to be.

      • MotoAsh@lemmy.world
        link
        fedilink
        English
        arrow-up
        4
        ·
        edit-2
        1 year ago

        I mean… those “typical dysfunctions” are what OP described. You’re just describing the general state of the industry, not providing further examples of disfunction.

        • grue@lemmy.world
          link
          fedilink
          English
          arrow-up
          4
          ·
          1 year ago

          No, the parent commenter attributed it to management not wanting to spend money. I’m attributing it to management being incompetent instead, which isn’t the same thing. Spending even unlimited amounts of money is not sufficient to fix what’s wrong.

  • TheGrandNagus@lemmy.world
    link
    fedilink
    English
    arrow-up
    39
    ·
    edit-2
    1 year ago

    OTA updates are very much a double-edged sword.

    Usually it’s good, but sometimes you get crap like this or when Tesla downgraded a bunch of Model S batteries when they saw they weren’t holding up as well as expected and wouldn’t line up with their battery warranty.

    E: and it’s insane to me that cars don’t have snapshots of previously installed firmware/other software. Or like an A/B partition style thing where you can revert if an update goes bad.

  • Wahots@pawb.social
    link
    fedilink
    English
    arrow-up
    29
    ·
    1 year ago

    I’m a tech enthusiast, but I increasingly am slowing my roll on all updates but security updates. The era of heavily QAing feature updates is ending, which is leading to some frankly annoying issues across the board, from car software, to phone and computer software. Even GPUs don’t seem to be immune from occasional bugs in the latest updates.

    Companies and orgs really need to dedicate more time to making sure updates are vetted and clean before releasing them. It might cost you some money up front, but it saves you PR, security issues, and hotfixes in the longer run.

    • floofloof@lemmy.ca
      link
      fedilink
      English
      arrow-up
      12
      ·
      edit-2
      1 year ago

      in the longer run.

      I have yet to work for a company where management cares about this. It’s always about what can be sold next week if we rush it enough. Or, more commonly, what was sold weeks ago without any consultation with the technical team.

      • Wahots@pawb.social
        link
        fedilink
        English
        arrow-up
        4
        ·
        1 year ago

        It’s a problem everywhere. Rush to build something, then don’t actually finish it, or iron out the bugs before moving on to the next sexy thing. Management wonders why everyone is on the super ancient, rocksolid platform and hasn’t really taken the bait on the series of products that have an “agile” development cycle and were dropped after about 5 years.

        Guys…my entire team knows the issue. Support your damn products for years. Commit to improvements for a decade, and help people transition to a new product when the old one has to be sunsetted for technological or knowledge worker retirement reasons.

    • Ebby@lemmy.ssba.com
      link
      fedilink
      English
      arrow-up
      7
      ·
      1 year ago

      I have a Sync 3 system that filled its self up with files and no longer has free memory. Its a software bug that was fixed, but I’d have to pay to replace the entire unit rather than snag a utility to clear space and upgrade. That is soooo dumb.

      But an unaffiliated community created a fix before Ford. Thanks hackers!

  • CADmonkey@lemmy.world
    link
    fedilink
    English
    arrow-up
    29
    arrow-down
    13
    ·
    1 year ago

    Weird. My primitive old Samurai started right up this morning, and it’s “infotainment” system (bluetooth boombox bungee-corded to the floor behind the seats) still works.

    Maybe if I’d paid $90,000 for it I could be privileged enough to have a broken functionality.

  • rikonium@discuss.tchncs.de
    link
    fedilink
    English
    arrow-up
    12
    ·
    1 year ago

    I think while the topic is up it’ll be fun mentioning that the Colorado/Canyon does not have a physical headlight control anymore - in favor of defaulting to Auto and touchscreen controls and the project lead(?) claimed that the system was 100% bug-free.

    Also later there was a bug with some OTA update for that model that’ll kill the battery.

    Anyways, I bought a 4Runner and immediately called to disable its cellular radio. (dubbed DCM in Toyota-land)

      • rikonium@discuss.tchncs.de
        link
        fedilink
        English
        arrow-up
        9
        ·
        1 year ago

        There have been some experimentation but the best (IMO) option is hitting the SOS button to call and navigating the phone menu to get to a representative, not emergency services and having them disconnect your car. You may need your VIN, you’ll need to confirm that you do not want connected services and it may take a day or so to take effect. Now, my SOS button doesn’t have the green light and while the radio - according to the infotainment - is still powered, it is no longer connected to the network.

        Another way if you don’t plan on using the microphone (like for calls) you can pull the DCM fuse but I prefer the above option.

    • Darkenfolk@dormi.zone
      link
      fedilink
      English
      arrow-up
      19
      arrow-down
      2
      ·
      1 year ago

      “According to Rivian the glitch occurred when the wrong build with incorrect security certificates was inadvertently sent out. This error resulted in bricking the infotainment systems of an unknown number of vehicles.”

      I always thought that ‘fat finger’ revered to having fingers that are too big for buttons, but I guess it means sending out wrong software versions.

      • LastYearsPumpkin@feddit.ch
        link
        fedilink
        English
        arrow-up
        24
        ·
        1 year ago

        Fat finger in this context typically means that the person knew what they wanted to do, but typed the wrong thing. Their “fat finger” hit the wrong button. Possible that they wanted certificate 123 in the build, but they accidentally typed 133, or something like that.

      • grue@lemmy.world
        link
        fedilink
        English
        arrow-up
        7
        arrow-down
        4
        ·
        1 year ago

        I think it’s also worth noting that “security certificates” in this context are almost certainly there for the purpose of locking the system away from modification by the vehicle owner, which means they weren’t necessary (and indeed, ought to be prohibited by law) to begin with.

        Rivian was hoist on their own consumer-hostile petard.

        • TrumpetX@programming.dev
          link
          fedilink
          English
          arrow-up
          3
          ·
          1 year ago

          I don’t think so in this context. This is probably more like SSL cert trusting or some private/public key pair.

          • LastYearsPumpkin@feddit.ch
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 year ago

            Based on the fact that the screen is blank, I think @grue was correct. The certificate that tells the system that the infotainment was valid was incorrect, so the infotainment couldn’t load. They used the dev-infotainment cert instead of the prod cert.

    • TurnItOff_OnAgain@lemmy.world
      link
      fedilink
      English
      arrow-up
      20
      arrow-down
      4
      ·
      1 year ago

      They did

      Rm -rf /

      To force remove everything recursively instead of

      Rm -fr /

      To remove the French language pack. Rookie mistake really.

    • dhork@lemmy.world
      link
      fedilink
      English
      arrow-up
      10
      arrow-down
      3
      ·
      1 year ago

      Dumb-ass managers who thought they could run their embedded software group all Agile and shit, and push stuff out to Production without full testing to meet deadlines, not realizing that they can’t just “roll back” this one.

    • coffeebiscuit@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      arrow-down
      2
      ·
      1 year ago

      They had 2 files in one directory. One said “good update” the other said “bad update-don’t use”. When they wanted to push out the update they used a tablet and tapped on “good update” but they accidentally touched the wrong one a bit more with the thumb.

      Something like this. But maybe they don’t want to say wat really happened.

  • Technus@lemmy.zip
    link
    fedilink
    English
    arrow-up
    6
    ·
    1 year ago

    Between the massive privacy issues and cars getting bricked by automatic OTA updates, I currently see zero reason whatsoever to buy a car made after ~2015.

    I just recently upgraded to a 2013 Rav4 and it has all the tech I could ever want. I’m gonna keep it as long as I can.