API call fail

[Ben00793]

Anyone else having issues today with API calls?
I'm in the UK.

[autoSteve]

Interesting. Nothing during the daylight hours AEST TZ UTC+10 and all forecast updates fine, but a re-start hit a 429 on sites load for me just now. Five additional re-starts also hit 429.

So yes, @Ben00793. It's them, not us.

(I think the false declaration of presumption of death has been fixed in v4.3.3, BTW. That rings a bell. The log below is from v4.3.3 code running on my prod instance.)

2025-04-28 22:04:38.299 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Configuration directory is /config
2025-04-28 22:04:38.299 DEBUG (MainThread) [custom_components.solcast_solar] Session headers: {'Accept': 'application/json', 'User-Agent': 'ha-solcast-solar-integration/4.3'}
2025-04-28 22:04:38.312 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Sites cache exists for ******NV8ggJ
2025-04-28 22:04:38.312 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Connecting to https://api.solcast.com.au/rooftop_sites?format=json&api_key=******NV8ggJ
2025-04-28 22:04:38.370 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned status 429/Try again later, trying cache
2025-04-28 22:04:38.370 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Get sites failed, last call result: 429/Try again later, using cached data
2025-04-28 22:04:38.370 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Loading cached sites for ******NV8ggJ
2025-04-28 22:04:38.371 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Checking rekey for ******NV8ggJ
2025-04-28 22:04:38.371 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Sites data: {'sites': [{'name': 'Iris North-East', 'resource_id': 'b68d-c05a-c2b3-2cf9', 'capacity': 6.857, 'capacity_dc': 7.92, 'azimuth': -67, 'tilt': 26, 'loss_factor': 0.98, 'tags': [], 'api_key': '******NV8ggJ'}, {'name': 'Iris North-West', 'resource_id': '83d5-ab72-2a9a-2397', 'capacity': 5.243, 'capacity_dc': 5.72, 'azimuth': 27, 'tilt': 17, 'loss_factor': 0.98, 'tags': [], 'api_key': '******NV8ggJ'}], 'page_count': 1, 'current_page': 1, 'total_records': 2}

[autoSteve]

Quickly altering the user agent string ('User-Agent': 'ha-solcast-solar-integration/4.3a') had no effect after HA restart, with it also getting a 429.

[autoSteve]

And significantly altering the user agent to ha-special-solar-integration/4.3a also saw 429.

I guess they're busy. Try again later. 🤞😉

This does not look Europe-specific, but I take it you're also hitting issues in the UK, @gcoan?

[gcoan]

@autoSteve my 5:34 and 9:16 auto updates both worked perfectly this morning first time for both sites with no errors

but the 12:58 retried 10 times up to 13:10, getting a 429 each time before it gave up

must be all the sun we are forecast to get this week, Solcast is throwing a wobbly because its not expected in the UK

[autoSteve]

So more than an hour later things are still all 429. At an almost on-the-hour restart, that is. Or six of them up to five minutes past the hour. Then at 18 minutes past the hour, 429 on restart.

I then wondered if this is not some scheduled maintenance or service fail. Nothing reported, at least. Just occasional hobbyist Hell I reckon. Maybe they turn us off in times of system stress? Who knows. 🤷‍♂️ We do get this high quality forecast for free.

Non sequitur: I am jealous of your sun, Geoff. Mine is failing me, heading for Winter, yet I still have not paid an electric bill since before September last year after joining a VPP. Today was a total crap <20kWh generation, yet I was still AUD 0.90 in credit despite using ~1kW 24x7 here. (In peak summer this credit is ~AUD 3.00 per good day. June/July/August are going to be really interesting, because if I can pull those off as a zero electric bill too then I have got everything sized just right and have found exactly the right retailer... And all this fancy solar+battery gubbins will pay itself off a lot earlier than the nine years I originally projected.)

[gcoan]

My 16:40 BST (15:40 UTC) scheduled update failed, same series of 429 errors.

Hopefully will come back sometime soon

Its amazing how the weather changes here in the UK, in January and February I was importing loads, heat pump running overnight. Now in April I will be about £120 in credit for the month with all the export I'm doing. I do have to trickle charge the batteries during the day though as I export so much it pushes the grid voltage up to 258V and shuts down the inverters. By charging and exporting it avoids this happening

[WyndStryke]

Now in April I will be about £120 in credit for the month with all the export I'm doing.

My array is much smaller than yours (15x 455W), installed in January, but generation has been pretty good in the last couple of months. My first export payment should be in the ballpark of £380 (quarterly, a combination of PV and arbitrage).

I coded it to do more overnight arbitrage when there is less PV forecast, and less if it is hot, or there is plenty of PV expected (by adjusting the charge and discharge rates).

[AndreR84]

Yes. Im living in Germany and i have the same Problem.

[WyndStryke]

My most recent scheduled (via an automation) call seems to have been missed, I don't have any logging in the automation (it just calls update). So I assume it is the same issue as reported here. It does one call in the early hours so that the overnight cheap rate import can be adjusted appropriately, and the other 4 calls are spread out evenly through the sunlit hours (east/west arrays so I get 5 rather than 10). Next & final call should happen in about an hour, fingers crossed that they resolve it at some point, the solcast forecasts are super useful :-)

Logbook
[Show more](https://xxxxxxxxxx.ui.nabu.casa/logbook?entity_id=sensor.solcast_pv_forecast_api_last_polled&start_date=2025-04-26T23%3A00%3A00.000Z&back=1)
28 April 2025
changed to 28 April 2025 at 09:49
09:49:37 - 6 hours ago
changed to 28 April 2025 at 06:23
06:23:15 - 9 hours ago
changed to 28 April 2025 at 01:14
01:14:32 - 15 hours ago
27 April 2025
changed to 27 April 2025 at 16:52
16:52:12 - Yesterday

I wonder if it might be worth having some sort of fall-back forecasting method if enough API calls fail in a row. A lot of my automations rely on it, and assume it'll always work, which is obviously a risky assumption.

[double1968]

Same problem in Italy

[gcoan]

I wonder if it might be worth having some sort of fall-back forecasting method if enough API calls fail in a row. A lot of my automations rely on it, and assume it'll always work, which is obviously a risky assumption.

if the forecast update fails the entities remain unchanged with the last forecast retrieved, its just less accurate. So no fallback needed.

And the forecast contains today and the next 6 days, obviously with a risk of being less accurate the further away, so you could go several days without an update occurring

[Nilogax]

Same here. I noticed repeated 429s a few weeks ago and contacted Solcast support who found a rogue account that was somehow hammering the API. Just emailed them again to see if this is something similar.

[billy-solcast]

Hey Team,

Just giving a quick update here, as a few of you have emailed in.

There was one user that caused the issue with excessive API requests. They've now been restricted, so the API should return to normal shortly.

[billy-solcast]

Hey team,

Thanks for getting back.
So quickly digging into it further, for the generic rooftop_sites, it looks like majority are from:
ha-solcast-solar-integration/4.3
ha-solcast-solar-integration/4.2

And digging further into that, there's someone sending through 4k's worth of API's (with ha-solcast-solar-integration/4.2) per day to /rooftop_sites.
Unfortunately I'll never have the time to chase all these instances of users, to free up the bandwidth for everyone, so it does make things difficult. As you say if it's an old version, not much we can do there.

For the incident that caused this outage, it wasn't a HA user. And it looks like all the other users that triggered the flow on impact weren't HA, so it looks like we're good there. The attemps (retry) from HA doesn't look that bad, unfortunately I can't provide more information, I'd probably still consider it maybe a tad high? Not sure how it works, by the looks of the top 30 users, sent approx 600 API request (each) in that one day period.

I'll probably leave that there. If there is any further insight you need that I can help with, please let me know.

[autoSteve]

there's someone sending through 4k's worth of API's (with ha-solcast-solar-integration/4.2) per day to /rooftop_sites.

That is surely an init abend going on, @billy-solcast. My theory: If sites get loaded as part of the init code, and then later in the init something causes a crash then Home Assistant will repeatedly try a restart.

/rooftop_sites is called in the following places.

1. On integration startup, despite the configured sites being cached. The cache is only used should the call hit 429/Too busy.

You'd think that little can be changed here because we need to accommodate adds/changes in Solcast configuration. But I have an optimisation idea.

A cache is always written with this call, so there's nothing in the rule book that says I can't detect whether startup has crashed prior, and then if so only use the cache until a non-crash restart occurs. This init crash-detection code is already there and used for another purpose, so extending its use should be super simple.

That won't help your current Mr/Ms 4.2.x's situation, but it will mitigate for any future init-crashing scenarios. You may be able to reach out to that account holder and encourage them to upgrade once I get some optimisations baked in. I'll target v4.3.3 for these (this release is imminent).

        try:
            self.sites = []

            api_keys = self.options.api_key.split(",")
            for api_key in api_keys:
                api_key = api_key.strip()
                cache_filename = self.__get_sites_cache_filename(api_key)
                _LOGGER.debug(
                    "%s",
                    f"Sites cache {'exists' if Path(cache_filename).is_file() else 'does not yet exist'}"
                    f" for {self.__redact_api_key(api_key)}",
                )
                url = f"{self.options.host}/rooftop_sites"
                params = {"format": "json", "api_key": api_key}
                _LOGGER.debug("Connecting to %s?format=json&api_key=%s", url, self.__redact_api_key(api_key))

                success = False
                cache_exists = Path(cache_filename).is_file()
                response: ClientResponse = await self._aiohttp_session.get(url=url, params=params, headers=self.headers, ssl=False)
                status = response.status

2. I forgot this one... Introduced in 4.3.0: Whenever a configuration change is made via the GUI to verify the API key, and then again should the config change cause an integration restart. This should be a rare occurrence, with the code being there to align with the Home Assistant 'Gold' standard.

Note that I am going to alter this code slightly for you guys, such that it only calls the sites validation should that be required (i.e. API key change).

    async def test_api_key(self) -> tuple[int, str]:
        """Test the API key. Used in the config flow."""
        api_keys = self.options.api_key.split(",")
        for api_key in api_keys:
            api_key = api_key.strip()
            url = f"{self.options.host}/rooftop_sites"
            params = {"format": "json", "api_key": api_key}
            _LOGGER.debug("Test connection to %s?format=json&api_key=%s", url, self.__redact_api_key(api_key))

            response: ClientResponse = await self._aiohttp_session.get(url=url, params=params, headers=self.headers, ssl=False)
            if response.status != 200:
                if response.status in (401, 403):
                    return response.status, f"Bad API key, {self.__translate(response.status)} returned for {api_key}"
                return response.status, f"Error {self.__translate(response.status)} for API key {api_key}"
            sites = await response.json()
            if sites.get("total_records") == 0:
                return 404, f"No sites found for API key {api_key}"
        self.reauth_required = False
        return 200, ""

[gcoan]

Other possibilities are:

Users of Predbat who installed the oziee integration before the instructions were updated to point to this repo - or users who installed it some time ago. Unlike if they have installed via HA, there may be little to no reason to update.

As well as the potential options for things that call the Solcast API that @BJReplay highlighted, there is another potential option, Predbat can and does directly call the Solcast API.

When the Oziee repo got nuked the author of Predbat added direct integration to the Solcast API's as an option to Predbat. The Predbat instructions explain the difference between using the BJreplay Solcast integration (more configurability, 429 auto retries, dampening, automatic sunrise to sunset API calls, PV10/50/90 attribute breakdowns), or Predbat calling Solcast directly (simple 'get the forecast').

So there is likely a fair number of Predbat users directly calling the Solcast API's.
The Predbat poll frequency is by default set to every 8 hours, and is measured in hours, so unlikely that its spamming the API's unless someone has set it to something stupid like 0.01 hours?

Predbat doesn't have anything like the caching and sophistication that the Solcast integration has, it's likely that it is calling solcast_sites every time it gets the forecast. Would you like me to raise a feature request to get this changed @billy-solcast ?

[autoSteve]

FYI, code tweaks are done. Just putting finishing touches on the PyTest, so this code should be available from tomorrow sometime in the v4.3.3 payload.

What will now happen when the integration is presumed dead is that a 999/Prior crash status will be set instead of attempting to call the API /rooftop_sites. This will prevent an integration when repeatedly started by HA when in a failed state from making any API calls after the first until it is fixed. If a restart is triggered by a human doing a RECONFIGURE or a REAUTH then presumption of death is set false immediately prior to that warm-body initiated restart, ensuring that sites will be loaded from an API call in that circumstance.

Session headers: {'Accept': 'application/json', 'User-Agent': 'ha-solcast-solar-integration/4.3'}
Sites cache exists for ******2
Get sites failed, last call result: 999/Prior crash, using cached data
Loading cached sites for ******2
Checking rekey for ******2
Sites data: {'sites': [{'resource_id': '3333-3333-3333-3333', 'name': 'Third Site', 'install_date': '2024-01-01T00:00:00+00:00', 'loss_factor': 0.99, 'capacity': 3.0, 'capacity_dc': 3.5, 'azimuth': 90, 'tilt': 30, 'location': 'Downunder', 'api_key': '******2'}], 'counter': 0, 'page_count': 1, 'current_page': 1, 'total_records': 1}

As for verifying an API key works during CONFIG I already had it rigged so it would only validate on change of API key.

[autoSteve]

FYI, PR #281 awaiting review prior to merge.

Tests updated to add this scenario, and 100% success and code coverage.

Job's done from my end.

(ha-venv) vscode ➜ /workspaces/…/tests/components/solcast_solar $ pytest --cov=homeassistant.components.solcast_solar --cov-report term-missing -v
Test session starts (platform: linux, Python 3.13.3, pytest 8.3.4, pytest-sugar 1.0.0)
cachedir: .pytest_cache
rootdir: /workspaces/homeAssistant-core
configfile: pyproject.toml
plugins: github-actions-annotate-failures-0.2.0, sugar-1.0.0, pytest_freezer-0.4.8, picked-0.5.0, syrupy-4.8.0, timeout-2.3.1, respx-0.22.0, socket-0.7.0, cov-6.0.0, aiohttp-1.0.5, xdist-3.6.1, asyncio-0.24.0, requests-mock-1.12.1, unordered-0.6.1, anyio-4.8.0
asyncio: mode=Mode.AUTO, default_loop_scope=function
collected 74 items                                                                                                                                                                                                     

 tests/components/solcast_solar/test_config_flow.py::test_single_instance ✓                                                                                                                                1% ▎         
 tests/components/solcast_solar/test_config_flow.py::test_create_entry ✓                                                                                                                                   3% ▍         
 tests/components/solcast_solar/test_config_flow.py::test_init_api_key[user_input0-api_looks_like_site] ✓                                                                                                  4% ▌         
 tests/components/solcast_solar/test_config_flow.py::test_init_api_key[user_input1-api_duplicate] ✓                                                                                                        5% ▋         
 tests/components/solcast_solar/test_config_flow.py::test_init_api_key[user_input2-None] ✓                                                                                                                 7% ▋         
 tests/components/solcast_solar/test_config_flow.py::test_init_api_key[user_input3-None] ✓                                                                                                                 8% ▊         
 tests/components/solcast_solar/test_config_flow.py::test_init_api_key[user_input4-None] ✓                                                                                                                 9% ▉         
 tests/components/solcast_solar/test_config_flow.py::test_init_api_key[user_input5-limit_one_or_greater] ✓                                                                                                11% █▏        
 tests/components/solcast_solar/test_config_flow.py::test_config_api_key_invalid ✓                                                                                                                        12% █▎        
 tests/components/solcast_solar/test_config_flow.py::test_config_api_quota[options0-user_input0-limit_not_number] ✓                                                                                       14% █▍        
 tests/components/solcast_solar/test_config_flow.py::test_config_api_quota[options1-user_input1-limit_one_or_greater] ✓                                                                                   15% █▌        
 tests/components/solcast_solar/test_config_flow.py::test_config_api_quota[options2-user_input2-limit_too_many] ✓                                                                                         16% █▋        
 tests/components/solcast_solar/test_config_flow.py::test_config_api_quota[options3-user_input3-None] ✓                                                                                                   18% █▊        
 tests/components/solcast_solar/test_config_flow.py::test_config_api_quota[options4-user_input4-None] ✓                                                                                                   19% █▉        
 tests/components/solcast_solar/test_config_flow.py::test_config_api_quota[options5-user_input5-limit_too_many] ✓                                                                                         20% ██        
 tests/components/solcast_solar/test_config_flow.py::test_config_api_quota[options6-user_input6-None] ✓                                                                                                   22% ██▎       
 tests/components/solcast_solar/test_config_flow.py::test_reauth_api_key[component.solcast_solar.config.error.Bad API key, 403/Forbidden returned for 555] ✓                                              23% ██▍       
 tests/components/solcast_solar/test_config_flow.py::test_reconfigure_api_key1 ✓                                                                                                                          24% ██▌       
 tests/components/solcast_solar/test_config_flow.py::test_reconfigure_api_key2[None-options0-Bad API key, 403/Forbidden-ignore_translations0] ✓                                                           26% ██▋       
 tests/components/solcast_solar/test_config_flow.py::test_reconfigure_api_key2[None-options1-No sites found for API key-ignore_translations1] ✓                                                           27% ██▊       
 tests/components/solcast_solar/test_config_flow.py::test_reconfigure_api_key2[return_429-options2-Error 429/Try again later for API key-ignore_translations2] ✓                                          28% ██▉       
 tests/components/solcast_solar/test_config_flow.py::test_reconfigure_api_key2[None-options3-None-ignore_translations3] ✓                                                                                 30% ███       
 tests/components/solcast_solar/test_config_flow.py::test_reconfigure_api_quota ✓                                                                                                                         31% ███▎      
 tests/components/solcast_solar/test_config_flow.py::test_options_api_key[user_input0-api_looks_like_site] ✓                                                                                              32% ███▎      
 tests/components/solcast_solar/test_config_flow.py::test_options_api_key[user_input1-api_duplicate] ✓                                                                                                    34% ███▍      
 tests/components/solcast_solar/test_config_flow.py::test_options_api_key[user_input2-None] ✓                                                                                                             35% ███▌      
 tests/components/solcast_solar/test_config_flow.py::test_options_api_key[user_input3-None] ✓                                                                                                             36% ███▋      
 tests/components/solcast_solar/test_config_flow.py::test_options_api_key[user_input4-None] ✓                                                                                                             38% ███▊      
 tests/components/solcast_solar/test_config_flow.py::test_options_api_key[user_input5-limit_one_or_greater] ✓                                                                                             39% ███▉      
 tests/components/solcast_solar/test_config_flow.py::test_options_api_key_invalid ✓                                                                                                                       41% ████▏     
 tests/components/solcast_solar/test_config_flow.py::test_options_api_quota[options0-user_input0-limit_not_number] ✓                                                                                      42% ████▎     
 tests/components/solcast_solar/test_config_flow.py::test_options_api_quota[options1-user_input1-limit_one_or_greater] ✓                                                                                  43% ████▍     
 tests/components/solcast_solar/test_config_flow.py::test_options_api_quota[options2-user_input2-limit_too_many] ✓                                                                                        45% ████▌     
 tests/components/solcast_solar/test_config_flow.py::test_options_api_quota[options3-user_input3-None] ✓                                                                                                  46% ████▋     
 tests/components/solcast_solar/test_config_flow.py::test_options_api_quota[options4-user_input4-None] ✓                                                                                                  47% ████▊     
 tests/components/solcast_solar/test_config_flow.py::test_options_api_quota[options5-user_input5-limit_too_many] ✓                                                                                        49% ████▉     
 tests/components/solcast_solar/test_config_flow.py::test_options_api_quota[options6-user_input6-None] ✓                                                                                                  50% █████     
 tests/components/solcast_solar/test_config_flow.py::test_options_custom_hour_sensor[options0-0-custom_invalid] ✓                                                                                         51% █████▎    
 tests/components/solcast_solar/test_config_flow.py::test_options_custom_hour_sensor[options1-145-custom_invalid] ✓                                                                                       53% █████▍    
 tests/components/solcast_solar/test_config_flow.py::test_options_custom_hour_sensor[options2-8-None] ✓                                                                                                   54% █████▌    
 tests/components/solcast_solar/test_config_flow.py::test_options_hard_limit[options0-invalid-hard_not_number] ✓                                                                                          55% █████▋    
 tests/components/solcast_solar/test_config_flow.py::test_options_hard_limit[options1--1-hard_not_number] ✓                                                                                               57% █████▋    
 tests/components/solcast_solar/test_config_flow.py::test_options_hard_limit[options2-6,6.0-hard_too_many] ✓                                                                                              58% █████▊    
 tests/components/solcast_solar/test_config_flow.py::test_options_hard_limit[options3-6-None] ✓                                                                                                           59% █████▉    
 tests/components/solcast_solar/test_config_flow.py::test_options_hard_limit[options4-6,6.0-None] ✓                                                                                                       61% ██████▏   
 tests/components/solcast_solar/test_config_flow.py::test_options_hard_limit[options5-6-None] ✓                                                                                                           62% ██████▎   
 tests/components/solcast_solar/test_config_flow.py::test_options_hard_limit[options6-0-None] ✓                                                                                                           64% ██████▍   
 tests/components/solcast_solar/test_config_flow.py::test_step_to_dampen ✓                                                                                                                                65% ██████▌   
 tests/components/solcast_solar/test_config_flow.py::test_dampen[value0] ✓                                                                                                                                66% ██████▋   
 tests/components/solcast_solar/test_config_flow.py::test_entry_options_upgrade ✓                                                                                                                         68% ██████▊   
 tests/components/solcast_solar/test_config_flow.py::test_presumed_dead_and_full_flow ✓                                                                                                                   69% ██████▉   
 tests/components/solcast_solar/test_diagnostics.py::test_diagnostics ✓                                                                                                                                   70% ███████   
 tests/components/solcast_solar/test_energy.py::test_energy_data ✓                                                                                                                                        72% ███████▎  
 tests/components/solcast_solar/test_forecast_retry.py::test_forecast_retry ✓                                                                                                                             73% ███████▍  
 tests/components/solcast_solar/test_integration.py::test_api_failure ✓                                                                                                                                   74% ███████▌  
 tests/components/solcast_solar/test_integration.py::test_schema_upgrade ✓                                                                                                                                76% ███████▋  
 tests/components/solcast_solar/test_integration.py::test_integration[options0] ✓                                                                                                                         77% ███████▊  
 tests/components/solcast_solar/test_integration.py::test_integration[options1] ✓                                                                                                                         78% ███████▉  
 tests/components/solcast_solar/test_integration.py::test_integration[options2] ✓                                                                                                                         80% ████████  
 tests/components/solcast_solar/test_integration.py::test_integration[options3] ✓                                                                                                                         81% ████████▎ 
 tests/components/solcast_solar/test_integration.py::test_integration_remaining_actions ✓                                                                                                                 82% ████████▎ 
 tests/components/solcast_solar/test_integration.py::test_integration_scenarios ✓                                                                                                                         84% ████████▍ 
 tests/components/solcast_solar/test_repairs.py::test_missing_data_fixable ✓                                                                                                                              85% ████████▌ 
 tests/components/solcast_solar/test_select.py::test_select_change_value[0-estimate-forecast_today-42.552] ✓                                                                                              86% ████████▋ 
 tests/components/solcast_solar/test_select.py::test_select_change_value[1-estimate10-forecast_today-35.46] ✓                                                                                             88% ████████▊ 
 tests/components/solcast_solar/test_select.py::test_select_change_value[2-estimate90-forecast_today-47.28] ✓                                                                                             89% ████████▉ 
 tests/components/solcast_solar/test_sensor.py::test_sensor_states[2-settings0] ✓                                                                                                                         91% █████████▏
 tests/components/solcast_solar/test_sensor.py::test_sensor_states[1-settings1] ✓                                                                                                                         92% █████████▎
 tests/components/solcast_solar/test_sensor.py::test_sensor_x_hours_long ✓                                                                                                                                93% █████████▍
 tests/components/solcast_solar/test_sensor.py::test_sensor_unavailable ✓                                                                                                                                 95% █████████▌
 tests/components/solcast_solar/test_sensor.py::test_sensor_unavailble_exception ✓                                                                                                                        96% █████████▋
 tests/components/solcast_solar/test_specific_times.py::test_midnight ✓                                                                                                                                   97% █████████▊
 tests/components/solcast_solar/test_specific_times.py::test_timezone_transition ✓                                                                                                                        99% █████████▉
 tests/components/solcast_solar/test_system_health.py::test_system_health ✓                                                                                                                              100% ██████████

---------- coverage: platform linux, python 3.13.3-final-0 -----------
Name                                                                                     Stmts   Miss  Cover   Missing
----------------------------------------------------------------------------------------------------------------------
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/__init__.py          409      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/config_flow.py       241      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/const.py              43      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/coordinator.py       290      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/diagnostics.py        17      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/energy.py             16      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/repairs.py            42      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/select.py             43      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/sensor.py            157      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/solcastapi.py       1467      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/system_health.py      11      0   100%
/workspaces/homeAssistant-core/homeassistant/components/solcast_solar/util.py              128      0   100%
----------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                     2864      0   100%


Results (25.83s):
      74 passed

[BJReplay]

@billy-solcast still no luck here nearly 2 hours later:

No updates at all today, including four or five since you indicated that it should start coming good and your status update on your website indicated that service has returned to normal https://status.solcast.com/incidents/288675.

That makes me think that you're still blocking legitimate requests.

Just in case it was my box, I've reloaded the integration:

2025-04-29 12:11:07.320 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Cancelling coordinator task listeners
2025-04-29 12:11:07.321 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Cancelling coordinator task check_fetch
2025-04-29 12:11:07.321 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Cancelling coordinator task midnight_update
2025-04-29 12:11:07.327 DEBUG (MainThread) [custom_components.solcast_solar] Remove action: solcast_solar.clear_all_solcast_data
2025-04-29 12:11:07.327 DEBUG (MainThread) [custom_components.solcast_solar] Remove action: solcast_solar.force_update_forecasts
2025-04-29 12:11:07.327 DEBUG (MainThread) [custom_components.solcast_solar] Remove action: solcast_solar.get_dampening
2025-04-29 12:11:07.327 DEBUG (MainThread) [custom_components.solcast_solar] Remove action: solcast_solar.query_forecast_data
2025-04-29 12:11:07.327 DEBUG (MainThread) [custom_components.solcast_solar] Remove action: solcast_solar.remove_hard_limit
2025-04-29 12:11:07.327 DEBUG (MainThread) [custom_components.solcast_solar] Remove action: solcast_solar.set_dampening
2025-04-29 12:11:07.327 DEBUG (MainThread) [custom_components.solcast_solar] Remove action: solcast_solar.set_hard_limit
2025-04-29 12:11:07.328 DEBUG (MainThread) [custom_components.solcast_solar] Remove action: solcast_solar.update_forecasts
2025-04-29 12:11:07.328 DEBUG (MainThread) [custom_components.solcast_solar] Removing presumed dead flag
2025-04-29 12:11:07.328 DEBUG (MainThread) [custom_components.solcast_solar] Auto-update options: {'auto_update': 1}
2025-04-29 12:11:07.328 DEBUG (MainThread) [custom_components.solcast_solar] Estimate to use options: {'key_estimate': 'estimate'}
2025-04-29 12:11:07.328 DEBUG (MainThread) [custom_components.solcast_solar] Attribute options: {'attr_brk_detailed': False, 'attr_brk_estimate': False, 'attr_brk_estimate10': True, 'attr_brk_estimate90': False, 'attr_brk_halfhourly': True, 'attr_brk_hourly': False, 'attr_brk_site': False}
2025-04-29 12:11:07.328 DEBUG (MainThread) [custom_components.solcast_solar] Custom sensor options: {'customhoursensor': 1}
2025-04-29 12:11:07.328 DEBUG (MainThread) [custom_components.solcast_solar] Hard limit: {'hard_limit_api': '100.0'}
2025-04-29 12:11:07.331 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Configuration directory is /config
2025-04-29 12:11:07.331 DEBUG (MainThread) [custom_components.solcast_solar] Session headers: {'Accept': 'application/json', 'User-Agent': 'ha-solcast-solar-integration/4.3'}
2025-04-29 12:11:07.357 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Sites cache exists for ******jonURa
2025-04-29 12:11:07.358 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Connecting to https://api.solcast.com.au/rooftop_sites?format=json&api_key=******jonURa
2025-04-29 12:11:07.816 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned status 429/Try again later, trying cache
2025-04-29 12:11:07.816 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Get sites failed, last call result: 429/Try again later, using cached data
2025-04-29 12:11:07.816 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Loading cached sites for ******jonURa
2025-04-29 12:11:07.818 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Checking rekey for ******jonURa
2025-04-29 12:11:07.818 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Sites data: {'sites': [{'name': 'BLAH', 'resource_id': 'a546-98c5-53c5-d101', 'capacity': 5, 'capacity_dc': 4.85, 'azimuth': -123, 'tilt': 5, 'install_date': '2012-04-18T14:00:00.0000000Z', 'loss_factor': 0.8, 'tags': ['Home', 'St Kilda West', 'Flat Roof', 'Home Assistant'], 'location': 'St Kilda West, VIC Australia', 'api_key': '******jonURa'}], 'page_count': 1, 'current_page': 1, 'total_records': 1}
2025-04-29 12:11:07.818 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Sites loaded
2025-04-29 12:11:07.819 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Usage cache exists for ******jonURa
2025-04-29 12:11:07.821 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Usage cache for ******jonURa last reset 2025-04-29 10:00:00
2025-04-29 12:11:07.821 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Usage loaded
2025-04-29 12:11:07.821 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] API counter for ******jonURa is 0/20
2025-04-29 12:11:07.822 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Using legacy hourly dampening
2025-04-29 12:11:07.870 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data cache /config/solcast.json exists, file type is <class 'dict'>
2025-04-29 12:11:07.870 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Dampened data loaded
2025-04-29 12:11:07.876 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data cache /config/solcast-undampened.json exists, file type is <class 'dict'>
2025-04-29 12:11:07.876 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Un-dampened data loaded
2025-04-29 12:11:07.978 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Task build_data took 0.100 seconds
2025-04-29 12:11:08.021 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecast data from 2025-04-29 to 2025-05-05 contains all intervals
2025-04-29 12:11:08.021 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecast data for 2025-05-06 contains 0 of 48 intervals, which may be expected
2025-04-29 12:11:08.025 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Task recalculate_splines took 0.004 seconds
2025-04-29 12:11:08.026 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Sun rise / set today: 06:59:30 / 17:35:02
2025-04-29 12:11:08.027 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Auto update total seconds: 38132, self.divisions: 20, interval: 1906 seconds
2025-04-29 12:11:08.027 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Auto update forecasts between sunrise and sunset
2025-04-29 12:11:08.027 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Previous auto update would have been at 11:45:29
2025-04-29 12:11:08.027 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Previous auto update UTC 2025-04-29T01:45:29+00:00
2025-04-29 12:11:08.027 INFO (MainThread) [custom_components.solcast_solar.coordinator] Auto forecast updates for today at 12:17:16, 12:49:02, 13:20:49, 13:52:35, 14:24:22, 14:56:09, 15:27:55, 15:59:42, 16:31:28, 17:03:15
2025-04-29 12:11:08.027 INFO (MainThread) [custom_components.solcast_solar.coordinator] Auto forecast updates for tomorrow at 07:00, 07:32, 08:03, 08:35, 09:07, 09:38, 10:10, 10:42, 11:13, 11:45, 12:17, 12:48, 13:20, 13:52, 14:23, 14:55, 15:27, 15:58, 16:30, 17:02
2025-04-29 12:11:08.027 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Running task check_fetch
2025-04-29 12:11:08.027 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Running task listeners
2025-04-29 12:11:08.027 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Running task midnight_update
2025-04-29 12:11:08.027 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Finished fetching solcast_solar data in 0.000 seconds (success: True)
2025-04-29 12:11:08.028 DEBUG (MainThread) [custom_components.solcast_solar] UTC times are converted to Australia/Sydney
2025-04-29 12:11:08.028 DEBUG (MainThread) [custom_components.solcast_solar] Successful init
2025-04-29 12:11:08.028 INFO (MainThread) [custom_components.solcast_solar] Solcast integration version: v4.3.2
2025-04-29 12:11:08.060 DEBUG (MainThread) [custom_components.solcast_solar] Checking whether auto update forecast is stale
2025-04-29 12:11:08.060 INFO (MainThread) [custom_components.solcast_solar] Last auto update forecast recorded (2025-04-28 17:04:20) is older than expected, should be (2025-04-29 11:45:29), updating forecast
2025-04-29 12:11:08.060 DEBUG (MainThread) [custom_components.solcast_solar] Register action: solcast_solar.clear_all_solcast_data
2025-04-29 12:11:08.060 DEBUG (MainThread) [custom_components.solcast_solar] Register action: solcast_solar.force_update_forecasts
2025-04-29 12:11:08.060 DEBUG (MainThread) [custom_components.solcast_solar] Register action: solcast_solar.get_dampening
2025-04-29 12:11:08.060 DEBUG (MainThread) [custom_components.solcast_solar] Register action: solcast_solar.query_forecast_data
2025-04-29 12:11:08.060 DEBUG (MainThread) [custom_components.solcast_solar] Register action: solcast_solar.remove_hard_limit
2025-04-29 12:11:08.061 DEBUG (MainThread) [custom_components.solcast_solar] Register action: solcast_solar.set_dampening
2025-04-29 12:11:08.061 DEBUG (MainThread) [custom_components.solcast_solar] Register action: solcast_solar.set_hard_limit
2025-04-29 12:11:08.061 DEBUG (MainThread) [custom_components.solcast_solar] Register action: solcast_solar.update_forecasts
2025-04-29 12:11:08.062 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Started task update_missed
2025-04-29 12:11:08.062 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Checking for stale usage cache
2025-04-29 12:11:08.062 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site a546-98c5-53c5-d101
2025-04-29 12:11:08.062 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site a546-98c5-53c5-d101, last day 2025-05-06, 180 hours
2025-04-29 12:11:08.062 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2025-04-29 12:11:08.123 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url https://api.solcast.com.au/rooftop_sites/a546-98c5-53c5-d101/forecasts?format=json&api_key=******jonURa&hours=180
2025-04-29 12:11:08.123 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Call status 429/Try again later, pausing 24 seconds before retry
2025-04-29 12:11:32.124 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2025-04-29 12:11:32.166 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url https://api.solcast.com.au/rooftop_sites/a546-98c5-53c5-d101/forecasts?format=json&api_key=******jonURa&hours=180
2025-04-29 12:11:32.166 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Call status 429/Try again later, pausing 32 seconds before retry
2025-04-29 12:12:04.167 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2025-04-29 12:12:04.207 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url https://api.solcast.com.au/rooftop_sites/a546-98c5-53c5-d101/forecasts?format=json&api_key=******jonURa&hours=180
2025-04-29 12:12:04.207 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Call status 429/Try again later, pausing 51 seconds before retry
2025-04-29 12:12:55.209 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2025-04-29 12:12:55.302 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url https://api.solcast.com.au/rooftop_sites/a546-98c5-53c5-d101/forecasts?format=json&api_key=******jonURa&hours=180
2025-04-29 12:12:55.302 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Call status 429/Try again later, pausing 72 seconds before retry

[billy-solcast]

I'm looking into it

[autoSteve]

Successful update just now...

[Ben00793]

Same, I was able to reload the integration just now. Let's see how the day goes l.

[BJReplay]

Same. 2:30-ish according to my logging - but I was busy doing other stuff, so didn't notice :)

[double1968]

Ok, work !

[autoSteve]

v4.3.3 is in the wild.

Enjoy.

(And thanks again for the feeedback @billy-solcast. 😘 The failed-state optimisation is in the v4.3.3 payload.)

[Nilogax]

Bumping this as I've had repeated 429s today since around 9am UK time. Anything you can see @billy-solcast?

[billy-solcast]

Was an individual user, whom sent through 1.23million API calls in the space of 7 hours. Then stopped.

The service looks to have returned to normal.

[BJReplay]

1.23million API calls in the space of 7 hours.

Nearly 50 per second - NDDOS (Non-Distributed)

[autoSteve]

It won't be, but call out as soon as possible if it was our user-agent string @billy-solcast.

Thanks for the update.

[billy-solcast]

It wasn't a Home-Assistant user, so all good. If it is, and if it's multiple people, I'll let you know. Generally I'd say these isolated users, aren't likely to be, as you guys have been great with the code maintenance!

[Ben00793]

Same here. Last successful update was at 9:55 UK time.

[BJReplay]

Just updated here in Aus for first sunrise update of the day.

Slept through the outage.

[tabascoz]

yep working now :) great ! think Solcast team were sleeping also lol

Cheers

[BJReplay]

@billy-solcast I've been getting 429s since 2025-05-23T08:19:31.671+10 (2025-05-22T22:19:31.671Z)

[billy-solcast]

Hey team,

Another user (non HA) spamming through API requests.
I've since restricted their access, and it looks like service is returning to normal.

[BJReplay]

Thanks, back again as of just before 1pm AEST.