RAND(Squawk): DNS fail causes strange behaviour

** Final Outcome as at 31 December 2024

It was rather annoying that we never actually got the very bottom of this problem. We had a full discovery exploration and examination of the problem but the best conclusion we could get to was something going wrong with the internal to the PC network software infrastructure. The external equipment router Wi-Fi, DNS servers was all proven to work during one of these outages by using a mobile phone (with mobile data service turned off) attached to the Wi-F. Was it asked having a temporary flip out? Was it something within Windows 10 ? - We never quite found out, which is rather annoying.

Since the trouble occurred at the beginning of December, we've have instigated the following two processes :

1) Reboot the pc every week

2) Reboot the router once a month.

It was interesting if for rather frustrating problem to explore. It's been a month of smooth running now for which we are very grateful.

The other routine procedure we do is plug-in an external hard drive and copy over the 10 most important business documents at the beginning of every month. This process supplements the file history mechanism built into Windows 10.

----------------

Original story

I do a bit of technical support for a small retail antique shop customer. Most of the most of the callouts are fairly routine but once in a while a strange little problem will appear. The symptoms of this issue are the inability of web browsers to reach webpages and sometimes Email connection failures (but not necessarily both at the same time). Firefox and Chrome are the two web browsers used to cross check web page connections.

Configuration is single BT broadband networking with BT Hub acting as a router. The connection from the PC to the hub is wired but the hub also provides a Wi-Fi service around the shop. There had been some account problems at BT but these been resolved. The PC has AVAST antivirus installed with a paid and up-to-date subscription.

When the particular occurs the Wi-Fi service still works correctly providing webpages and app updates. This would indicate that the problem is within the PC or connection to the hub. To confirm that the problem is directly within the PC the old-fashioned command line tools nslookup and ping were used. Also, the hub has a direct IP address to connect to obtain its status.

When the fault is *not* apparent both nslookup and ping work correctly.

During a failure the nslookup fails but the ping to an IP address works.

The AVST software thinks there is a total network failure but there is not. ( ping works )

During this time the Email service continues, probably because it has done the DNS look ups that are needed for the service during the time that DNS is working.

During a failure situation the windows network diagnostics indicates a DNS service failure. This diagnosis matches the behaviour (ping works) above and as such rules out total network connection failure.

Most telling the AVST service shows an ability to monitor DNS/DoH scanning.

This was disabled as part of the problem resolution process.

These are the network adapters showing. The AVAST Secureline is infrastructure for an unused VPN service.

When the system is rebooted, the network services typically recover but fail again within a few hours or days. But what we have here is an example of a specific DNS service failure. Either something is interfering with the DNS service or the DNS service itself is not functioning properly. I would think it's unlikely to be the latter because the Wi-Fi service continues unaffected during the time that this fault is occurring this lead to the conclusion that something on the machine itself is interfering with the DNS service.

As of 14 Oct 2024

The remediation process was to set the DNS servers for the network adapter in use to be the gold standard 8.8.8.8 (Google main service) and as back up 208.67.222.222 from OpenDNS. Also the DNS/DoH scanning was disabled ( as shown above). At this point, we are awaiting a recurrence of the issue. If the issue does not re occur, we will reenable DNS/DoH scanning and if after that the problem reoccurs we will be phoning up Avast fairly quickly to show them the problem we have.

The useful suggestion was made to use the ipconfig command to gather more details.

As of 21 October 2024

After a week of normal running, the same situation arose on Saturday morning. DNS had failed completely, but the network connections were still up as evidence by being able to ping an IP address. At this point IP config/was collected. The situation seem to be remedy by IP config/renew. However, the fix did not last long and soon failed with the same symptoms again. Only a full reboot restored normal service.

ifconfig while fault was occurring ....

C:\Users\pbpc>ipconfig /all

Windows IP Configuration

Host Name . . . . . . . . . . . . : PB
Primary Dns Suffix . . . . . . . :
Node Type . . . . . . . . . . . . : Mixed
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : home

Unknown adapter SecureLine:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Avast SecureLine Wintun Adapter
Physical Address. . . . . . . . . :
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Ethernet adapter Ethernet:

Connection-specific DNS Suffix . : home
Description . . . . . . . . . . . : Realtek PCIe GbE Family Controller
Physical Address. . . . . . . . . : 44-8A-5B-CB-FC-93
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 2a00:23c7:9c13:6b01:9639:1294:b919:fb95(Preferred)
Temporary IPv6 Address. . . . . . : 2a00:23c7:9c13:6b01:387e:c001:19a9:bde1(Preferred)
Link-local IPv6 Address . . . . . : fe80::f6c7:9835:4ccd:6755%9(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.1.65(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Lease Obtained. . . . . . . . . . : 16 October 2024 09:30:17
Lease Expires . . . . . . . . . . : 20 October 2024 09:29:56
Default Gateway . . . . . . . . . : fe80::8e83:94ff:fe6f:6902%9
192.168.1.254
DHCP Server . . . . . . . . . . . : 192.168.1.254
DHCPv6 IAID . . . . . . . . . . . : 138709595
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1B-FE-6A-E1-44-8A-5B-CB-FC-93
DNS Servers . . . . . . . . . . . : fe80::8e83:94ff:fe6f:6902%9
8.8.8.8
208.67.222.222
fe80::8e83:94ff:fe6f:6902%9
NetBIOS over Tcpip. . . . . . . . : Enabled
Connection-specific DNS Suffix Search List :
home

C:\Users\pbpc>nslookup hp.com
DNS request timed out.
timeout was 2 seconds.
Server: UnKnown
Address: fe80::8e83:94ff:fe6f:6902
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
*** Request to UnKnown timed-out

C:\Users\pbpc>ping 17.253.144.10
Pinging 17.253.144.10 with 32 bytes of data:
Reply from 17.253.144.10: bytes=32 time=16ms TTL=57
Reply from 17.253.144.10: bytes=32 time=16ms TTL=57
Reply from 17.253.144.10: bytes=32 time=16ms TTL=57
Reply from 17.253.144.10: bytes=32 time=16ms TTL=57
Ping statistics for 17.253.144.10:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 16ms, Maximum = 16ms, Average = 16ms
C:\Users\pbpc>

After the ifconfig /renew

C:\Users\pbpc>ipconfig /all

Windows IP Configuration

Host Name . . . . . . . . . . . . : PB
Primary Dns Suffix . . . . . . . :
Node Type . . . . . . . . . . . . : Mixed
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : home

Unknown adapter SecureLine:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Avast SecureLine Wintun Adapter
Physical Address. . . . . . . . . :
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Ethernet adapter Ethernet:

Connection-specific DNS Suffix . : home
Description . . . . . . . . . . . : Realtek PCIe GbE Family Controller
Physical Address. . . . . . . . . : 44-8A-5B-CB-FC-93
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 2a00:23c7:9c13:6b01:9639:1294:b919:fb95(Preferred)
Temporary IPv6 Address. . . . . . : 2a00:23c7:9c13:6b01:387e:c001:19a9:bde1(Preferred)
Link-local IPv6 Address . . . . . : fe80::f6c7:9835:4ccd:6755%9(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.1.65(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Lease Obtained. . . . . . . . . . : 16 October 2024 09:30:18
Lease Expires . . . . . . . . . . : 20 October 2024 12:19:01
Default Gateway . . . . . . . . . : fe80::8e83:94ff:fe6f:6902%9
192.168.1.254
DHCP Server . . . . . . . . . . . : 192.168.1.254
DHCPv6 IAID . . . . . . . . . . . : 138709595
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1B-FE-6A-E1-44-8A-5B-CB-FC-93
DNS Servers . . . . . . . . . . . : fe80::8e83:94ff:fe6f:6902%9
8.8.8.8
208.67.222.222
fe80::8e83:94ff:fe6f:6902%9
NetBIOS over Tcpip. . . . . . . . : Enabled
Connection-specific DNS Suffix Search List :
home

C:\Users\pbpc>ping apple.com

Pinging apple.com [2620:149:af0::10] with 32 bytes of data:
Reply from 2620:149:af0::10: time=15ms
Reply from 2620:149:af0::10: time=15ms
Reply from 2620:149:af0::10: time=15ms
Reply from 2620:149:af0::10: time=16ms

Ping statistics for 2620:149:af0::10:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 15ms, Maximum = 16ms, Average = 15ms

C:\Users\pbpc>ping apple.com

Pinging apple.com [2620:149:af0::10] with 32 bytes of data:
Reply from 2620:149:af0::10: time=15ms
Reply from 2620:149:af0::10: time=15ms
Reply from 2620:149:af0::10: time=15ms
Reply from 2620:149:af0::10: time=15ms

Ping statistics for 2620:149:af0::10:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 15ms, Maximum = 15ms, Average = 15ms

C:\Users\pbpc>tracert 172.16.15.245

Tracing route to 172.16.15.245 over a maximum of 30 hops

1 1 ms <1 ms <1 ms 192.168.1.254
2 * * * Request timed out.
3 * * * Request timed out.
4 * ^C
C:\Users\pbpc>tracert 172.16.15.245
^C
C:\Users\pbpc>ping 172.16.15.245

Pinging 172.16.15.245 with 32 bytes of data:
Request timed out.
Request timed out.

Ping statistics for 172.16.15.245:
Packets: Sent = 2, Received = 0, Lost = 2 (100% loss),
Control-C
^C
C:\Users\pbpc>ping apple.com

Pinging apple.com [2620:149:af0::10] with 32 bytes of data:
Reply from 2620:149:af0::10: time=38ms
Reply from 2620:149:af0::10: time=15ms
Reply from 2620:149:af0::10: time=15ms
Reply from 2620:149:af0::10: time=15ms

Ping statistics for 2620:149:af0::10:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 15ms, Maximum = 38ms, Average = 20ms

C:\Users\pbpc>tracert apple.com

Tracing route to apple.com [2620:149:af0::10]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 2a00:23c7:9c13:6b01:8e83:94ff:fe6f:6902
2 15 ms 14 ms 14 ms 2a00:2302::1102:203:50a
3 * * * Request timed out.
4 13 ms 13 ms 13 ms 2a00:2302::1102:100:3f
5 15 ms 16 ms * 2a00:2380:3014:9000::26
6 15 ms 16 ms 19 ms peer8-et0-1-5.telehouse.ukcore.bt.net [2a00:2380:14::77]
7 14 ms 13 ms 13 ms 2a00:2380:2001:8000::2d
8 15 ms 15 ms 15 ms icloud.com [2620:149:af0::10]

Trace complete.

C:\Users\pbpc>nslookup bbc.com
DNS request timed out.
timeout was 2 seconds.
Server: UnKnown
Address: fe80::8e83:94ff:fe6f:6902

DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
*** Request to UnKnown timed-out

C:\Users\pbpc>nslookup apple.com
DNS request timed out.
timeout was 2 seconds.
Server: UnKnown
Address: fe80::8e83:94ff:fe6f:6902

DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
*** Request to UnKnown timed-out

C:\Users\pbpc>

After a reboot at 13:30

Microsoft Windows [Version 10.0.19045.5011]
(c) Microsoft Corporation. All rights reserved.

C:\Users\pbpc>ipconfig /all

Windows IP Configuration

Host Name . . . . . . . . . . . . : PB
Primary Dns Suffix . . . . . . . :
Node Type . . . . . . . . . . . . : Mixed
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : home

Unknown adapter SecureLine:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Avast SecureLine Wintun Adapter
Physical Address. . . . . . . . . :
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes

Ethernet adapter Ethernet:

Connection-specific DNS Suffix . : home
Description . . . . . . . . . . . : Realtek PCIe GbE Family Controller
Physical Address. . . . . . . . . : 44-8A-5B-CB-FC-93
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes
IPv6 Address. . . . . . . . . . . : 2a00:23c7:9c13:6b01:9639:1294:b919:fb95(Preferred)
Temporary IPv6 Address. . . . . . : 2a00:23c7:9c13:6b01:5833:476e:d0ba:a1e(Preferred)
Link-local IPv6 Address . . . . . : fe80::f6c7:9835:4ccd:6755%9(Preferred)
IPv4 Address. . . . . . . . . . . : 192.168.1.65(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Lease Obtained. . . . . . . . . . : 19 October 2024 13:14:39
Lease Expires . . . . . . . . . . : 20 October 2024 13:14:39
Default Gateway . . . . . . . . . : fe80::8e83:94ff:fe6f:6902%9
192.168.1.254
DHCP Server . . . . . . . . . . . : 192.168.1.254
DHCPv6 IAID . . . . . . . . . . . : 138709595
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1B-FE-6A-E1-44-8A-5B-CB-FC-93
DNS Servers . . . . . . . . . . . : fe80::8e83:94ff:fe6f:6902%9
8.8.8.8
208.67.222.222
fe80::8e83:94ff:fe6f:6902%9
NetBIOS over Tcpip. . . . . . . . : Enabled
Connection-specific DNS Suffix Search List :
home

Wireless LAN adapter WiFi:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Qualcomm Atheros AR5BWB222 Wireless Network Adapter
Physical Address. . . . . . . . . : 30-10-B3-F1-5F-68
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes

Wireless LAN adapter Local Area Connection* 10:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft Hosted Network Virtual Adapter
Physical Address. . . . . . . . . : 52-10-B3-F1-5F-68
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes

Wireless LAN adapter Local Area Connection* 11:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft Wi-Fi Direct Virtual Adapter
Physical Address. . . . . . . . . : 12-10-B3-F1-5F-68
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes

Wireless LAN adapter Local Area Connection* 12:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Microsoft Wi-Fi Direct Virtual Adapter #2
Physical Address. . . . . . . . . : 22-10-B3-F1-5F-68
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes

Ethernet adapter Bluetooth Network Connection:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Bluetooth Device (Personal Area Network)
Physical Address. . . . . . . . . : 30-10-B3-F1-76-10
DHCP Enabled. . . . . . . . . . . : Yes
Autoconfiguration Enabled . . . . : Yes

The only difference in the Ipconfig output between before and after reboots is the Temporary IPv6 Address

% diff ipc_1.txt ipc_2.txt

29c29

< Temporary IPv6 Address. . . . . . : 2a00:23c7:9c13:6b01:387e:c001:19a9:bde1(Preferred)

---

> Temporary IPv6 Address. . . . . . : 2a00:23c7:9c13:6b01:5833:476e:d0ba:a1e(Preferred)

33,34c33,34

< Lease Obtained. . . . . . . . . . : 16 October 2024 09:30:17

< Lease Expires . . . . . . . . . . : 20 October 2024 09:29:56

---

> Lease Obtained. . . . . . . . . . : 19 October 2024 13:14:39

> Lease Expires . . . . . . . . . . : 20 October 2024 13:14:39

Still a bit stuck on this one. A cloud of suspicion hangs over the BT business hub 2 that is supposed to be providing network DNS services. However AVAST that does IP monitoring and provides VPN services brackets (not used brackets but still running processes the background) remains under that same cloud of suspicion.

Looks very similar Symptoms and occurrence rate to this issue "Smart Hub 2. Have to restart at least once a week because DNS Servers stop responding."

Avast VPN processes even thought the VPN option is switched off and inactive.

Notes on How DNS works on Windows

From the documentation:

https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/dd197552%28v=ws.10%29?redirectedfrom=MSDN

The DNS Client service queries the DNS servers in the following order:

The DNS Client service sends the name query to the first DNS server on the preferred adapter’s list of DNS servers and waits one second for a response.
If the DNS Client service does not receive a response from the first DNS server within one second, it sends the name query to the first DNS servers on all adapters that are still under consideration and waits two seconds for a response.
If the DNS Client service does not receive a response from any DNS server within two seconds, the DNS Client service sends the query to ALL DNS servers on ALL adapters that are still under consideration and waits another two seconds for a response.
If the DNS Client service still does not receive a response from any DNS server, it sends the name query to all DNS servers on all adapters that are still under consideration and waits four seconds for a response.
If it the DNS Client service does not receive a response from any DNS server, the DNS client sends the query to all DNS servers on all adapters that are still under consideration and waits eight seconds for a response.

If the DNS Client service receives a positive response, it stops querying for the name, adds the response to the cache and returns the response to the client.

If the DNS Client service has not received a response from any server within eight seconds, the DNS Client service responds with a timeout. Also, if it has not received a response from any DNS server on a specified adapter, then for the next 30 seconds, the DNS Client service responds to all queries destined for servers on that adapter with a timeout and does not query those servers.

If at any point the DNS Client service receives a negative response from a server, it removes every server on that adapter from consideration during this search. For example, if in step 2, the first server on Alternate Adapter A gave a negative response, the DNS Client service would not send the query to any other server on the list for Alternate Adapter A.

The DNS Client service keeps track of which servers answer name queries more quickly, and it moves servers up or down on the list based on how quickly they reply to name queries.

--------------------------------------------------------------------

This for fun ..... but seriously .. it matches this exact situation

.. and these are our exact symptoms, contact to specific IP address is fine but resolution to a name fails.

RAND(Squawk)

Monday, 14 October 2024

DNS fail causes strange behaviour

** Final Outcome as at 31 December 2024

Original story

As of 14 Oct 2024

As of 21 October 2024

No comments: