CRC cannot be started anymore mac #4513

rmestrum · 2024-12-13T13:43:31Z

General information

Unable to start CRC on a MAC macOS version 15.1.1

When starting I get the following errors

INFO Using bundle path /Users/rmestrum/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle 
INFO Checking if running macOS version >= 13.x    
INFO Checking if running as non-root              
INFO Checking if crc-admin-helper executable is cached 
INFO Checking if running on a supported CPU architecture 
INFO Checking if crc executable symlink exists    
INFO Checking minimum RAM requirements            
INFO Check if Podman binary exists in: /Users/rmestrum/.crc/bin/oc 
INFO Checking if running emulated on Apple silicon 
INFO Checking if vfkit is installed               
INFO Checking if old launchd config for tray and/or daemon exists 
INFO Checking if crc daemon plist file is present and loaded 
INFO Checking SSH port availability               
INFO Loading bundle: crc_vfkit_4.17.7_arm64...    
INFO Creating CRC VM for OpenShift 4.17.7...      
INFO Generating new SSH key pair...               
INFO Generating new password for the kubeadmin user 
INFO Starting CRC VM for openshift 4.17.7...      
INFO CRC instance is running with IP 127.0.0.1    
INFO CRC VM is running                            
INFO Updating authorized keys...                  
INFO Resizing /dev/vda4 filesystem                
INFO Configuring shared directories               
INFO Check internal and public DNS query...       
WARN Failed public DNS query from the cluster: ssh command error:
command : curl --head quay.io
err     : Process exited with status 6
 :  
INFO Check DNS query from host...                 
INFO Verifying validity of the kubelet certificates... 
INFO Starting kubelet service                     
INFO Waiting for kube-apiserver availability... [takes around 2min] 
INFO Adding user's pull secret to the cluster...  
INFO Updating SSH key to machine config resource... 
INFO Waiting until the user's pull secret is written to the instance disk... 
Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x2)
failed to read /var/lib/kubelet/config.json file: ssh command error:
command : sudo cat /var/lib/kubelet/config.json
err     : Process exited with status 1
: /bin/bash: Input/output error

Operating System

macOS

Hypervisor

vfkit

Did you run `crc setup` before `crc start`?

yes

Running on

Laptop

Steps to reproduce

Download the latest version of CRC (2.45.0+7aeb3b in my case)
Run crc setup
Run crc start -p

CRC version

❯ crc version  
CRC version: 2.45.0+7aeb3b
OpenShift version: 4.17.7
MicroShift version: 4.17.7

CRC status

❯ crc status                                      
CRC VM:          Running
OpenShift:       Unreachable (v4.17.7)
RAM Usage:       0B of 0B
Disk Usage:      -1B of -1B (Inside the CRC VM)
Cache Usage:     38.87GB
Cache Directory: /Users/rmestrum/.crc/cache

CRC config

❯ crc config view             
- consent-telemetry                     : no
- disk-size                             : 100
- memory                                : 14000

Host Operating System

❯ sw_vers          
ProductName:		macOS
ProductVersion:		15.1.1
BuildVersion:		24B91

Expected behavior

CRC starts and runs so it can be used.

Actual behavior

CRC starts gives error and CRC is not usable

CRC Logs

https://gist.github.com/rmestrum/9cb779c0c94770e7c81940d3794f760f

Additional context

No response

The text was updated successfully, but these errors were encountered:

cfergeau · 2024-12-13T13:52:47Z

DEBU non-retriable error: failed to read /var/lib/kubelet/config.json file: ssh command error:
command : sudo cat /var/lib/kubelet/config.json
err     : Process exited with status 1
: /bin/bash: Input/output error

This error is worrying, could be disk corruption, or maybe the cached bundle? Can you check you have enough free disk space? If you do, can you remove ~/.crc/cache/crc_vfkit_4.17.7_arm64, run crc cleanup and try again to use crc setup && crc start?

rmestrum · 2024-12-13T14:04:08Z

I already cleanup everything in my opinion but I followed the instructions and yes I have enough diskspace

Here is the output:

❯ rm -rf ~/.crc/cache/crc_vfkit_4.17.7_arm64 
❯ ls -al ~/.crc/cache                   
.rw-rw-r-- 5.3G rmestrum  6 Dec 07:09  crc_vfkit_4.17.7_arm64.crcbundle
❯ crc cleanup                                                       
INFO Unloading and removing the daemon plist file 
INFO Removing /etc/resolver/testing file          
INFO Stopping CRC vfkit process                   
INFO Removing CRC Specific entries from user's known_hosts file 
INFO Removing hosts file records added by CRC     
INFO Removing pull secret from the keyring        
INFO Removing older logs                          
Cleanup finished

❯ crc setup                                                         
INFO Using bundle path /Users/rmestrum/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle 
INFO Checking if running macOS version >= 13.x    
INFO Checking if running as non-root              
INFO Checking if crc-admin-helper executable is cached 
INFO Checking if running on a supported CPU architecture 
INFO Checking if crc executable symlink exists    
INFO Creating symlink for crc executable          
INFO Checking minimum RAM requirements            
INFO Check if Podman binary exists in: /Users/rmestrum/.crc/bin/oc 
INFO Checking if running emulated on Apple silicon 
INFO Checking if vfkit is installed               
INFO Checking if CRC bundle is extracted in '$HOME/.crc' 
INFO Checking if /Users/rmestrum/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle exists 
INFO Getting bundle for the CRC executable        
INFO Downloading bundle: /Users/rmestrum/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle... 
0 B / 4.94 GiB [______________________________________________________________________________________________________________________________________________________________________________________________________________________________] 0.00% ??/s
INFO Uncompressing /Users/rmestrum/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle 
crc.img:  31.00 GiB / 31.00 GiB [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
oc:  125.37 MiB / 125.37 MiB [-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00%
INFO Checking if old launchd config for tray and/or daemon exists 
INFO Checking if crc daemon plist file is present and loaded 
INFO Adding crc daemon plist file and loading it  
INFO Checking SSH port availability               
Your system is correctly setup for using CRC. Use 'crc start' to start the instance

❯ crc start -p /Users/rmestrum/CRC/pull-secret.txt                  
INFO Using bundle path /Users/rmestrum/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle 
INFO Checking if running macOS version >= 13.x    
INFO Checking if running as non-root              
INFO Checking if crc-admin-helper executable is cached 
INFO Checking if running on a supported CPU architecture 
INFO Checking if crc executable symlink exists    
INFO Checking minimum RAM requirements            
INFO Check if Podman binary exists in: /Users/rmestrum/.crc/bin/oc 
INFO Checking if running emulated on Apple silicon 
INFO Checking if vfkit is installed               
INFO Checking if old launchd config for tray and/or daemon exists 
INFO Checking if crc daemon plist file is present and loaded 
INFO Checking SSH port availability               
INFO Loading bundle: crc_vfkit_4.17.7_arm64...    
INFO Creating CRC VM for OpenShift 4.17.7...      
INFO Generating new SSH key pair...               
INFO Generating new password for the kubeadmin user 
INFO Starting CRC VM for openshift 4.17.7...      
INFO CRC instance is running with IP 127.0.0.1    
INFO CRC VM is running                            
INFO Updating authorized keys...                  
INFO Resizing /dev/vda4 filesystem                
INFO Configuring shared directories               
INFO Check internal and public DNS query...       
INFO Check DNS query from host...                 
INFO Verifying validity of the kubelet certificates... 
INFO Starting kubelet service                     
INFO Waiting for kube-apiserver availability... [takes around 2min] 
INFO Adding user's pull secret to the cluster...  
INFO Updating SSH key to machine config resource... 
INFO Waiting until the user's pull secret is written to the instance disk... 
Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x3)
failed to read /var/lib/kubelet/config.json file: ssh command error:
command : sudo cat /var/lib/kubelet/config.json
err     : Process exited with status 1
: sudo: unable to execute /bin/cat: Input/output error

jmanteau · 2024-12-15T10:29:08Z

I also have the same error on 14.7.1 (and I have 200GB free)

❯ crc cleanup
INFO Unloading and removing the daemon plist file
INFO Removing /etc/resolver/testing file
INFO Stopping CRC vfkit process
INFO Removing CRC Specific entries from user's known_hosts file
INFO Removing hosts file records added by CRC
INFO Removing pull secret from the keyring
INFO Removing older logs
Cleanup finished


❯  crc setup
INFO Using bundle path /Users/to108637/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle
INFO Checking if running macOS version >= 13.x
INFO Checking if running as non-root
INFO Checking if crc-admin-helper executable is cached
INFO Checking if running on a supported CPU architecture
INFO Checking if crc executable symlink exists
INFO Creating symlink for crc executable
INFO Checking minimum RAM requirements
INFO Check if Podman binary exists in: /Users/to108637/.crc/bin/oc
INFO Checking if running emulated on Apple silicon
INFO Checking if vfkit is installed
INFO Checking if CRC bundle is extracted in '$HOME/.crc'
INFO Checking if /Users/to108637/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle exists
INFO Checking if old launchd config for tray and/or daemon exists
INFO Checking if crc daemon plist file is present and loaded
INFO Adding crc daemon plist file and loading it
INFO Checking SSH port availability
Your system is correctly setup for using CRC. Use 'crc start' to start the instance


❯ crc start
INFO Using bundle path /Users/to108637/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle
INFO Checking if running macOS version >= 13.x
INFO Checking if running as non-root
INFO Checking if crc-admin-helper executable is cached
INFO Checking if running on a supported CPU architecture
INFO Checking if crc executable symlink exists
INFO Checking minimum RAM requirements
INFO Check if Podman binary exists in: /Users/to108637/.crc/bin/oc
INFO Checking if running emulated on Apple silicon
INFO Checking if vfkit is installed
INFO Checking if old launchd config for tray and/or daemon exists
INFO Checking if crc daemon plist file is present and loaded
INFO Checking SSH port availability
INFO Loading bundle: crc_vfkit_4.17.7_arm64...
CRC requires a pull secret to download content from Red Hat.
You can copy it from the Pull Secret section of https://console.redhat.com/openshift/create/local.
? Please enter the pull secret *****************************************************************************************

INFO Creating CRC VM for OpenShift 4.17.7...
INFO Generating new SSH key pair...
INFO Generating new password for the kubeadmin user
INFO Starting CRC VM for openshift 4.17.7...
INFO CRC instance is running with IP 127.0.0.1
INFO CRC VM is running
INFO Updating authorized keys...
INFO Configuring shared directories
INFO Check internal and public DNS query...
INFO Check DNS query from host...
INFO Verifying validity of the kubelet certificates...
INFO Starting kubelet service
INFO Waiting for kube-apiserver availability... [takes around 2min]
INFO Adding user's pull secret to the cluster...
INFO Updating SSH key to machine config resource...
INFO Waiting until the user's pull secret is written to the instance disk...
Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x19)```

2manoj1 · 2025-01-03T09:51:31Z

Same issue in Mac M1
OS version: 14.6.1

CRC version: 2.45.0
Any solution?

praveenkumar · 2025-01-06T05:09:34Z

When you folks hit this issue can use https://crc.dev/engineering-docs/Debugging.html#access-the-vm and see if able to ssh to the VM and if /var/lib/kubelet/config.json file present?

2manoj1 · 2025-01-06T09:43:58Z

I followed that document, still no luck.
Setup 12GB vRAM
60 GB disk

Secret copy and paste.

Anything else we need to do?

praveenkumar · 2025-01-06T11:11:21Z

I followed that document, still no luck.

Please be specific, what do you mean by no luck? does ssh to VM failed or config.json file was not present?

2manoj1 · 2025-01-06T11:57:29Z

I follow -
Download latest crc in Macos.
Install it.

Run below command
crc setup
crc start

Also tried crc start -p ~/Downloads/pull-secret.txt

praveenkumar · 2025-01-06T12:00:26Z

Did you try #4513 (comment) what I have asked?

2manoj1 · 2025-01-06T12:05:53Z

yes.

praveenkumar · 2025-01-07T05:45:13Z

Looks like we forgot to update our docs around :(, can you execute ssh -i ~/.crc/machines/crc/id_ed25519 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 2222 [email protected]

Also do you have docker-desktop/podman-desktop/minikube running on this system?

2manoj1 · 2025-01-07T08:04:35Z

yes podman desktop on my system

I tried ssh -i ~/.crc/machines/crc/id_ed25519 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -p 2222 [email protected]
Warning: Identity file /Users/mm/.crc/machines/crc/id_ed25519 not accessible: No such file or directory.
ssh: connect to host 127.0.0.1 port 2222: Connection refused

crc stop --force
crc cleanup
crc setup
crc start

INFO Loading bundle: crc_vfkit_4.17.7_arm64...
CRC requires a pull secret to download content from Red Hat.
You can copy it from the Pull Secret section of https://console.redhat.com/openshift/create/local.
? Please enter the pull secret *****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************

INFO Creating CRC VM for OpenShift 4.17.7...
INFO Generating new SSH key pair...
INFO Generating new password for the kubeadmin user
INFO Starting CRC VM for openshift 4.17.7...
INFO CRC instance is running with IP 127.0.0.1
INFO CRC VM is running
INFO Updating authorized keys...
INFO Resizing /dev/vda4 filesystem
INFO Configuring shared directories
INFO Check internal and public DNS query...
INFO Check DNS query from host...
INFO Verifying validity of the kubelet certificates...
INFO Starting kubelet service
INFO Waiting for kube-apiserver availability... [takes around 2min]
INFO Adding user's pull secret to the cluster...
INFO Updating SSH key to machine config resource...
INFO Waiting until the user's pull secret is written to the instance disk...
Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x2)
failed to read /var/lib/kubelet/config.json file: ssh command error:
command : sudo cat /var/lib/kubelet/config.json
err : Process exited with status 1
: /bin/bash: Input/output error

praveenkumar · 2025-01-08T05:34:56Z

yes podman desktop on my system

@2manoj1 Can you try following and see if that fixes it?

Stop the podman machine either from podman desktop UI or from cli podman machine stop .
crc delete => crc cleanup => crc setup => crc start

2manoj1 · 2025-01-08T07:28:07Z

@praveenkumar I tried but getting same error. I stop podman and follow suggested step. In hard disk 220 GB free. 16GB RAM Apple MAC M1 Pro. OS version - 14.6.1

~ % podman machine stop
Machine "podman-machine-default" stopped successfully

~ % crc delete
Machine does not exist. Use 'crc start' to create it
~ % crc cleanup
INFO Unloading and removing the daemon plist file
INFO Removing /etc/resolver/testing file
INFO Stopping CRC vfkit process
INFO Removing CRC Specific entries from user's known_hosts file
INFO Removing hosts file records added by CRC
INFO Removing pull secret from the keyring
INFO Removing older logs
Cleanup finished
~ % crc setup
INFO Using bundle path /Users/mm/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle
INFO Checking if running macOS version >= 13.x
INFO Checking if running as non-root
INFO Checking if crc-admin-helper executable is cached
INFO Checking if running on a supported CPU architecture
INFO Checking if crc executable symlink exists
INFO Creating symlink for crc executable
INFO Checking minimum RAM requirements
INFO Check if Podman binary exists in: /Users/mm/.crc/bin/oc
INFO Checking if running emulated on Apple silicon
INFO Checking if vfkit is installed
INFO Checking if CRC bundle is extracted in '$HOME/.crc'
INFO Checking if /Users/mm/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle exists
INFO Checking if old launchd config for tray and/or daemon exists
INFO Checking if crc daemon plist file is present and loaded
INFO Adding crc daemon plist file and loading it
INFO Checking SSH port availability
Your system is correctly setup for using CRC. Use 'crc start' to start the instance
~ % crc start
INFO Using bundle path /Users/mm/.crc/cache/crc_vfkit_4.17.7_arm64.crcbundle
INFO Checking if running macOS version >= 13.x
INFO Checking if running as non-root
INFO Checking if crc-admin-helper executable is cached
INFO Checking if running on a supported CPU architecture
INFO Checking if crc executable symlink exists
INFO Checking minimum RAM requirements
INFO Check if Podman binary exists in: /Users/mm/.crc/bin/oc
INFO Checking if running emulated on Apple silicon
INFO Checking if vfkit is installed
INFO Checking if old launchd config for tray and/or daemon exists
INFO Checking if crc daemon plist file is present and loaded
INFO Checking SSH port availability
INFO Loading bundle: crc_vfkit_4.17.7_arm64...
CRC requires a pull secret to download content from Red Hat.
You can copy it from the Pull Secret section of https://console.redhat.com/openshift/create/local.
? Please enter the pull secret ************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
INFO Creating CRC VM for OpenShift 4.17.7...
INFO Generating new SSH key pair...
INFO Generating new password for the kubeadmin user
INFO Starting CRC VM for openshift 4.17.7...

INFO CRC instance is running with IP 127.0.0.1
INFO CRC VM is running
INFO Updating authorized keys...
INFO Resizing /dev/vda4 filesystem
INFO Configuring shared directories
INFO Check internal and public DNS query...
INFO Check DNS query from host...
INFO Verifying validity of the kubelet certificates...
INFO Starting kubelet service
INFO Waiting for kube-apiserver availability... [takes around 2min]
INFO Adding user's pull secret to the cluster...
INFO Updating SSH key to machine config resource...
INFO Waiting until the user's pull secret is written to the instance disk...
Failed to update pull secret on the disk: Temporary error: pull secret not updated to disk (x15)
failed to read /var/lib/kubelet/config.json file: ssh command error:
command : sudo cat /var/lib/kubelet/config.json

praveenkumar · 2025-01-08T07:42:33Z

@2manoj1 Thanks for trying out and providing the info. Can you try one more thing instead of using OCP try using microshift preset and see if that works for you and ssh connectivity still working.

crc delete
crc cleanup
crc config set preset microshift
crc setup
crc start

2manoj1 · 2025-01-08T09:52:42Z

@praveenkumar
microshift is install and start. OpenShift having issue. how to access microshift? I don't have idea microshift

praveenkumar · 2025-01-08T10:10:53Z

https://www.redhat.com/en/topics/edge-computing/microshift explain what is microshift and if you want to consume it then kubeconfig file already added for same so you can run oc get pods -A and other queries using oc command.

tylercreller · 2025-01-17T16:29:26Z

FYI nuking docker from my laptop completely resolved my issues here. Only using the podman binary now.

MacOS: Sequoia 15.2 (24C101)

rmestrum · 2025-01-21T10:41:47Z

I check my mac and I don't have docker installed. I'll update to macos 15.2 soon

rmestrum added the kind/bug Something isn't working label Dec 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CRC cannot be started anymore mac #4513

CRC cannot be started anymore mac #4513

rmestrum commented Dec 13, 2024

cfergeau commented Dec 13, 2024

rmestrum commented Dec 13, 2024

jmanteau commented Dec 15, 2024 •

edited

Loading

2manoj1 commented Jan 3, 2025

praveenkumar commented Jan 6, 2025

2manoj1 commented Jan 6, 2025

praveenkumar commented Jan 6, 2025

2manoj1 commented Jan 6, 2025

praveenkumar commented Jan 6, 2025

2manoj1 commented Jan 6, 2025 •

edited

Loading

praveenkumar commented Jan 7, 2025 •

edited

Loading

2manoj1 commented Jan 7, 2025 •

edited

Loading

praveenkumar commented Jan 8, 2025

2manoj1 commented Jan 8, 2025

praveenkumar commented Jan 8, 2025

2manoj1 commented Jan 8, 2025

praveenkumar commented Jan 8, 2025

tylercreller commented Jan 17, 2025 •

edited

Loading

rmestrum commented Jan 21, 2025

CRC cannot be started anymore mac #4513

CRC cannot be started anymore mac #4513

Comments

rmestrum commented Dec 13, 2024

General information

Operating System

Hypervisor

Did you run crc setup before crc start?

Running on

Steps to reproduce

CRC version

CRC status

CRC config

Host Operating System

Expected behavior

Actual behavior

CRC Logs

Additional context

cfergeau commented Dec 13, 2024

rmestrum commented Dec 13, 2024

jmanteau commented Dec 15, 2024 • edited Loading

2manoj1 commented Jan 3, 2025

praveenkumar commented Jan 6, 2025

2manoj1 commented Jan 6, 2025

praveenkumar commented Jan 6, 2025

2manoj1 commented Jan 6, 2025

praveenkumar commented Jan 6, 2025

2manoj1 commented Jan 6, 2025 • edited Loading

praveenkumar commented Jan 7, 2025 • edited Loading

2manoj1 commented Jan 7, 2025 • edited Loading

praveenkumar commented Jan 8, 2025

2manoj1 commented Jan 8, 2025

praveenkumar commented Jan 8, 2025

2manoj1 commented Jan 8, 2025

praveenkumar commented Jan 8, 2025

tylercreller commented Jan 17, 2025 • edited Loading

rmestrum commented Jan 21, 2025

Did you run `crc setup` before `crc start`?

jmanteau commented Dec 15, 2024 •

edited

Loading

2manoj1 commented Jan 6, 2025 •

edited

Loading

praveenkumar commented Jan 7, 2025 •

edited

Loading

2manoj1 commented Jan 7, 2025 •

edited

Loading

tylercreller commented Jan 17, 2025 •

edited

Loading