UserAgent.me

What Does Your User Agent Say About You?

Archive

A user agent is a computer program representing a person, for example, a browser in a Web context.

Besides a browser, a user agent could be a bot scraping webpages, a download manager, or another app accessing the Web. Along with each request they make to the server, browsers include a self-identifying User-Agent HTTP header called a user agent (UA) string. This string often identifies the browser, its version number, and its host operating system.

Spam bots, download managers, and some browsers often send a fake UA string to announce themselves as a different client. This is known as user agent spoofing.

The user agent string can be accessed with JavaScript on the client side using the navigator.userAgent property.

A typical user agent string looks like this: "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:35.0) Gecko/20100101 Firefox/35.0".

(Source: Mozilla.org)

User Agent String

Browser Data

User Agent Client Hints

author
Bob Roberts
• Wednesday, 07 October, 2020
• 28 min read

Ideally, this header would give servers the ability to perform content negotiation, sending down exactly those bits that best represent the requested resource in a given user agent, optimizing both bandwidth and user experience. In practice, however, this header’s value exposes far more information about the user ’s device than seems appropriate as a default, on the one hand, and intentionally obscures the true user agent in order to bypass misguided server-side heuristics, on the other.

agent client hints detection device future luca passani cto
(Source: www.scientiamobile.com)

Contents

Individual vendors have taken stabs at altering their user agent strings, and have run into a few categories of feedback from developers that have stymied historical approaches: Brand and version information (e.g. “Chrome 69”) allows websites to work around known bugs in specific releases that aren’t otherwise detectable.

For example, implementations of Content Security Policy have varied wildly between vendors, and it’s difficult to know what policy to send in an HTTP response without knowing what browser is responsible for its parsing and execution. Developers will often negotiate what content to send based on the user agent and platform.

Similarly to #1, OS revisions and architecture can be responsible for specific bugs which can be worked around in website’s code, and narrowly useful for things like selecting appropriate executables for download (32 vs 64 bit, ARM vs Intel, etc). Sophisticated developers use model/make to tailor their sites to the capabilities of the device (e.g. ) and to pinpoint performance bugs and regressions which sometimes are specific to model/make.

The request header field gives a server information about the device on which a given user agent is executing. The request header field gives a server information about the platform on which a given user agent is executing.

The request header field gives a server information about the platform version on which a given user agent is executing. The request header field gives a server information about an useragent's branding and version.

(Source: hibi-update.org)

It is considered low entropy because it includes only the useragent's branding information, and the significant version number (both of which are fairly clearly sniffable by “examining the structure of other headers and by testing for the availability and semantics of the features introduced or modified between releases of a particular browser” ). The request header field gives a server information about whether an user agent prefers a “mobile” user experience.

It is considered low entropy because it is a single bit of information directly controllable by the user. Note: The high-entropy portions of the user agent information are retrieved through a Promise, in order to give user agents the opportunity to gate their exposure behind potentially time-consuming checks (e.g. by asking the user for their permission).

Collect pairs of brands and which represent the user agent, its equivalence class and/or its rendering engine. Append additional items to list containing objects, initialized with arbitrary and combinations.

Note: One approach to minimize caching variance when generating these random components could be to determine them at build time, and keep them identical throughout the lifetime of the user agent's significant version. This means that user agent information will not be leaked over plaintext channels, reducing the opportunity for network attackers to build a profile of a given agent ’s behavior over time.

The primary goal of UserAgentClientHints is to reduce the default entropy available to the network for passive fingerprinting. User agents ought to exercise judgement before granting access to this information, and MAY impose restrictions above and beyond the secure transport and delegation requirements noted above.

(Source: http2.try-and-test.net)

For instance, user agents could choose to reveal only on requests it intends to download, giving the server the opportunity to serve the right binary. The header, however, is likely to be impossible to remove entirely in the near-term, as existing sites' content negotiation code will continue to require its presence (see for a recent example of a new browser’s struggles in this area).

History has shown us that there are real incentives for user agents to lie about their branding in order to thread the needle of sites' sniffing scripts, and prevent their users from being blocked by UA-based allow/block lists. Resetting expectations may help to prevent abuse of the UA string’s brand in the short term, but probably won’t help in the long run.

User agents brands containing more than a single entry could encourage standardized processing of the UA string. By randomly including additional, intentionally incorrect, comma-separated entries with arbitrary ordering, they would reduce the chance that we ossify on a few required strings.

In order to encourage sites to rely on equivalence classes based on Chromium versions rather than exact UA sniffing, Chrome might remove itself from the set entirely. Browsers based on Chromium may use a similar UA string, but use their own brand as part of the set, enabling sites to count them.

When adding arbitrary values to brands, user agents MUST make sure that receivers of the header adhere to Structured Header parsing, by adding escaped double-quotes, commas and semi-colons to those values. The purpose of this is to make non-compliant server implementations immediately aware that their parsing code is inadequate.

(Source: hibi-update.org)

Note: One approach to minimize caching variance could be to determine the GREASE parts of the UA set at build time, and keep them identical throughout the lifetime of the useragent's significant version. Restricting user -land JavaScript code from influencing and modifying UA-CH headers has various security related advantages.

These cases exist, and will continue to be supported, but explicitly: the server will need to send an Accept-CH header to request this information. It's been supported in Chrome on desktop & Android since 2015, and other Chromium-based browsers, though it's not yet available in Firefox or Safari.

It's better to serve a default version of your content, and treat client hints as progressive enhancement, to be used when available but not depended upon. The exact lifetime is left up to the client (a previous draft included an Accept-CH-Lifetime header, but that's now been removed) but it's likely to be at least the rest of the current browser session.

This is a general prefix for a forbidden header name as defined by the Fetch spec. In addition to potential data transfer concerns (especially for HTTP/1.1 clients or servers), requesting too much information may trip a privacy budget or otherwise trigger permission prompts in the future, and implies collecting unnecessary personal information about your users.

The draft also proposes a navigator.userAgentData JavaScript API to access this hint data client -side, but that doesn't seem to be implemented anywhere yet. Right now, the only browser that supports this is Chrome, and only in the dev & canary channels, and behind a flag.

(Source: qiita.com)

That said, testing it out now allows you to see how this might impact your application, and start to see how you can capture any hints that you need to handle this, once this does land for real. To test this out today, you'll need an HTTPS server locally where you can log requests and play with the response headers, or you can use an HTTP debugger like HTTP Toolkit to directly inspect & inject responses to test around with.

Watch out for more changes in the same direction too, as browsers move other finger printable data behind client hints in the future, including the Accept-Language header, and begin investigating approaches like GREASE to mitigate sniffing risks. You can follow the detailed progress on this in the Chromium and Firefox bug trackers.

It is unstructured, so parsing it results in unnecessary complexity, which is often the cause for bugs and site compatibility issues that hurt users. These issues also disproportionately hurt users of less common browsers, as sites may have failed to test against their configuration.

The server can choose to vary its responses, for example by serving images at an appropriate resolution. There are ongoing discussions on enabling ClientHints on an initial request, but you should be considering responsive design or progressive enhancement before going down this route.

Caution: These properties are more complex than just a single value, so Structured Headers are used for representing lists and Boolean. Response Accept-CH Requestheader RequestExample valueDescription UASec-CH-UA"Chromium";v="84","Google Chrome";v="84" List of browser brands and their significant version.

(Source: www.buytechnow.co.uk)

UA-MobileSec-CH-UA-Mobile?1 Boolean indicating if the browser is on a mobile device (?1 for true) or not (?0 for false). While this may not be relevant to displaying the page, the site may want to offer a download which defaults to the right format.

The default Sec-CH-UA and Sec-CH-UA-Mobile header information can be accessed via the brands and mobile properties, respectively: However, if another Accept-CH header is received then that will completely replace the current hints the browser is sending.

This will return a string with the historical entries for compatibility reasons, but with sanitized specifics. I’ve been following the development of this initiative from Google more or less since it was first filed as a note on GitHub.

From Googles perspective it may seem easy to switch to the alternative UA-CH, but this is where the team pushing this change doesn’t understand the impact: Functionality based on device detection is critical, widespread and not only in front end code.

Is it the right approach that two people can push for such a big, potentially breaking change? 3-4 documents aimed at the web dev community on GitHub with high level explanations of functionality is not good enough.

client hints value case use caniuse source
(Source: deviceatlas.com)

At such a tight schedule, the tools, docs and infrastructure should work and be available for the community to play around with. Chrome Canary (v81) already send the sec-ch-ua header by default, but still with the defects outlined above, and the body of work that many stakeholders need to undertake, I think the schedule is too aggressive.

If adopted client hints will require a few simple changes for a website to continue to gain access to information about the device, browser, and operating system. It is almost identical to the techniques already needed to fully identify web browsers such as Opera or UC.

Client hints could remove much of the complexity associated with identifying the browser, operating system, and device. Client hints do not remove the need to maintain an accurate catalog of the associated device, browser, and operating system data.

Algorithms developed by 51Degrees can turn this single string of characters into information about the device (Pixel 2 XL), operating system (Android 8), and browser (Chrome 79) with over 99.9% accuracy. Such information is essential to analyze web traffic accurately, diagnose browser or device-specific issues such as slow performance of video rendering, and to adapt the website to the specific capabilities of the device.

It’s not clear how a mobile is defined and by whom which is why this business prefers Sec-CH-Model to enable the web service to decide. They can be requested either via an HTTP response header or via the HTML element.

(Source: developers.microad.co.jp)

RTT provides the approximate Round Trip Time, in milliseconds, on the application layer. The value of RTT is rounded to the nearest 25 milliseconds to prevent fingerprinting.

Down link expressed in megabits per second (Mbps), reveals the approximate downstream speed of the user's connection. Sec-CH-UA represents the user agent's brand and major version.

Sec-CH-UA-Mobile represents whether the user agent should receive a specifically «mobile» UX. This header's value has grown in both length and complexity over the years; a complicated dance between server-side sniffing to provide the right experience for the right devices on the one hand, and client -side spoofing in order to bypass incorrect or inconvenient sniffing on the other.

Safari has recently taken some steps to reduce the entropy of the user agent string, initially locking it to a single value, period, and then backing off a bit due to developer feedback. From a developer's standpoint, the detail available in the UA string is valuable, and they reasonably object to dropping it.

The feedback to Safari's UA string freeze was right along these lines, noting four broad categories of use: Brand and version information (e.g. “Chrome 69”) allows websites to work around known bugs in specific releases that aren't otherwise detectable.

hints client use
(Source: www.imghaste.com)

For example, implementations of Content Security Policy have varied wildly between vendors, and it's difficult to know what policy to send in an HTTP response without knowing what browser is responsible for its parsing and execution. Developers will often negotiate what content to send based on the user agent and platform.

Similarly to #1, OS revisions and architecture can be responsible for specific bugs which can be worked around in website's code, and narrowly useful for things like selecting appropriate executables for download (32 vs 64 bit, ARM vs Intel, etc). Sophisticated developers use model/make to tailor their sites to the capabilities of the device (e.g. Facebook Year Class) and to pinpoint performance bugs and regressions which sometimes are specific to model/make.

By default, servers should receive only the useragent's brand and significant version number. The user agent can make reasonable decisions about how to respond to sites' requests for more granular data.

We can ratchet this deprecation over time, beginning by freezing the version numbers in the header, then removing platform and model information as developers migrate to the alternative mechanisms proposed below. The Sec-CH-UA header field represents the user agent's brand and significant version.

Note: See the GREASE-like discussion below for how we could anticipate the inevitable lies which user agents might want to tell in this field. The Sec-CH-UA-Arch header field represents the underlying architecture's instruction set and width.

hints client use global support
(Source: www.imghaste.com)

The Sec-CH-UA-Model header field represents the user agent's underlying device model. The Sec-CH-UA-Mobile header field represents whether the user agent should receive a specifically “mobile” UX.

We could conceivably even inject a permission prompt between the site's request and the Promise's resolution, if we decided that was a reasonable approach. User agents will attach the Sec-CH-UA header to every secure outgoing request by default, with a value that includes only the significant version (e.g. “Chrome"; v="69").

Note the word “secure” in the paragraph above, and the SecureContext attribute in the IDL: these client hints will not be delivered to plaintext endpoints. In any event, moving to this opt-in model means that the extent of a site's usage can be monitored and evaluated.

Similarly, when serving JavaScript to users, one can avoid transpiration (which can result in bloat and inefficient code) for browsers that support the latest ES features that were used. For that use case to work, the server needs to be aware of the browser and its meaningful version, and map that to a list of available features.

Services that wish to do that using UA-CH will need to inspect the Sec-CH-UA header, that is sent by default on every request, and modify their response based on that. Sites that wish to provide market share analytics using UA-CH will need to inspect the Sec-CH-UA header, that is sent by default on every request, and keep a record of it.

hints client cloudinary
(Source: cloudinary.com)

Other reasons are slightly less legitimate (e.g. warning users that the site's developers hasn't tested in their browser). For that to work, the server needs to be aware, at HTML serving time, whether the user is on a mobile device or not.

If the dimension on which the split is made is memory, the Device-Memory Client Hint can be used to make that distinction. While, again, progressive enhancement can be used to modify those links using script, rather than bake them into the HTML, some sites may prefer server-side adaptation.

If they require platform and its version, they'd have to opt-in for those hints, or use client -side scripts to control the experimentation. For those notifications to be meaningful, sites need to recognize and communicate the commercial brand of the browser to the user.

These messages often also include the platform and its version in order to make sure the user knows which device is in question. The right binary executable for the current user depends on a few factors: their operating system, its version, as well as their CPU architecture.

In some environments, proxy servers may be used to verify that the different users accessing information are not doing so from obsolete devices, that are potential vulnerable to security issues. Those services will have to use the lower entropy values available through Sec-CH-UA for logging purposes, or opt-in to receive higher-entropy hints.

hints client headers moment stop think there
(Source: davidwalsh.name)

On the other hand, when specific issues are encountered, it may make sense for those services to opt-in to receive more details on the user agent, or use the userAgentData.getHighEntropyData() API for that purpose. We hope that alternative methods or APIs will exist to address the spam filtering and bot detection use cases in the future, as browsers may decide to intervene on behalf of their users by limiting the collection of user -identifying entropy (e.g., the Privacy Budget proposal).

Like the case of “spam filtering”, it would still be feasible to actively collect all the hints about the user as bits of entropy. Unlike the above case, this is something that proposals such as the Privacy Budget aim to prevent, without providing any alternative mechanisms for persistent user tracking.

Our goal should eventually be to ratchet down on some of this granularity as well, and my intuition is that we'll be able to do that more cleanly if we adjust the UA string in one fell swoop, and then move on to the rest rather than doubling back at some point in the future. That may break things in the future if we diverge significantly from today's behavior in some interesting way, but that doesn't seem like a risk unique to this proposal.

Complementary to that, user agents might encourage standardized processing of the UA string by randomly including additional, intentionally incorrect, comma-separated entries with arbitrary ordering (similar conceptually to TLS's GREASE). In order to encourage sites to rely on equivalence classes based on Chromium versions rather than exact UA sniffing, Chrome might remove itself from the set entirely.

Browsers based on Chromium may use a similar UA string, but use their own brand as part of the set, enabling sites to count them. We'd reflect this value in the navigator.userAgentData.brands attribute, which returns an array of dictionaries containing brand and version.

(Source: developers.microad.co.jp)

“Cries"; v="73" (or “Chrome on iOS", v="73", or similar) which is basically what's sent today, and categorizes the browser as distinct. Based on some discussion in w3ctag/design-reviews#320, it seems reasonable to forbid access to these headers from JavaScript, and demarcate them as browser-controlled clienthints, so they can be documented and included in requests without triggering CORS preflights.

This is currently implicitly defined in most modern browsers because they have two distinct UIs, a “desktop” version (i.e. Windows, macOS, etc.) It's also worth pointing out that most modern browsers also have an explicit “request desktop site” UI element in their mobile versions which should be honored.

In a more general sense, though, a “mobile” experience could be seen as a UX designed with smaller screens and touch-based interface in mind. In particular, attempts to restructure the string in-place would result in a lot of breakages of legacy apps.

Client hints are a set of opt-in HTTP request headers that give us insight into these aspects of the user ’s device and the network they’re connected to. By tapping into this information server side, we can change how we deliver content based on device and/or network conditions.

It describes what content types the browser understands, which the server can use to negotiate the response. Like Accept, client hints are another avenue for negotiating content, but in the context of device capabilities and network conditions.

browsers detect parsing string agent without mobile user medium hints client majority visits approach effective chrome become internet since makes
(Source: medium.com)

In this guide, we’ll describe all the available hints and some ways you can use them to make content delivery more accommodating to users. Unlike the Accept header, client hints don’t just magically appear (except Save-Data, which we’ll discuss later).

When the client reads this header, it’s being told “this site wants the Viewport-Width and Down link client hints.” Don’t worry about the specific hints themselves. There’s also an optional Accept-CH-Lifetime header which specifies the length of time, in seconds, the browser should remember the value you set for Accept-CH for your origin.

Note: Client hints don’t kick in on the navigation request the first time a user visits your site. Client hints describe one of two things: the device your users, well, use, and the network they’re using to access your site.

Density-corrected intrinsic size: the dimensions of a media resource after it has been corrected for pixel density. It’s the image’s intrinsic size divided by a device pixel ratio.

Let’s say you have an element that loads an image with a density-corrected intrinsic size of 320×240, but it also has CSS width and height properties with values of 256px and 192px applied to it, respectively. This hint can be used with other screen-specific hints to deliver different treatments (i.e., crops) of an image which are optimal for specific screen sizes (i.e., art direction), or to omit resources that are unnecessary for the current screen width.

hints client
(Source: www.scientiamobile.com)

This hint is useful when selecting image sources which correspond to a screen's pixel density (like x descriptors do in the secret attribute). For example, let’s say a user requests a page with a 320 CSS pixel wide screen with a Dr of 2.

In this case, the client is hinting to the server that an optimal intrinsic width for the requested image would be 85% of the viewport width (272 pixels) multiplied by the screen’s Dr (2), which equals 544 pixels. This hint is especially powerful because it not only takes into account the density-corrected width of the screen, but also reconciles this critical piece of information with the image’s extrinsic size within the layout.

This gives servers the opportunity to negotiate image responses that are optimal for both the screen and the layout. In cases where both the Dr and Width headers are in play, the extrinsic size of a resource can produce scenarios where the two differ.

When a Contender request header is sent, the browser will know how to scale the given image for the screen’s device pixel ratio and the layout. With them, we have the ability to tailor experiences to users by changing how we deliver resources to clients on slow connections.

Note: Network hint values are predictions based on past latency and bandwidth readings. Note: The value of RTT is rounded to the nearest 25 milliseconds to prevent fingerprinting. This hint is useful because of the role latency plays in loading performance.

user agents spoofing
(Source: www.fastvue.co)

Note: The value of Down link is rounded to the nearest multiple of 25 kilo bits per second. Because again, fingerprinting. In conjunction with RTT, Down link can be useful in changing how content is delivered to users based on the quality of a network connection.

Rather, it analyzes the current connection’s latency and bandwidth and determines what network profile it resembles most. Save-Data isn’t so much a hint describing network conditions as it is a user preference stating that pages should send fewer data.

While and secret are undeniably awesome tools, they can be time-consuming to develop and maintain for complex use cases. If applicable to your workflow, first select an image treatment (i.e., art-directed imagery) by checking the Viewport-Width hint.

Select an image resolution by checking the Width hint and the Dr hint, and choosing a source that fits the image’s layout size and screen density (similar to how x and w descriptors work in secret). Where my fictitious timber company client was concerned, I developed a naïve responsive image selection routine in PHP that uses client hints.

Client hints make it possible to start with a lossless, high-resolution image that can then be dynamically resized to be optimal for any combination of screen and layout. Unlike secret, which requires you to enumerate a fixed list of possible image candidates for the browser to choose from, this approach can be more flexible.

chrome google farewell agent client bugs hints strings bids welcome memory incognito working go apps fixes browser update latest windowsreport
(Source: www.qubicgroup.com)

Where Connie Timber’s site is concerned, we take steps to lighten the load when networks are slow, with Save-Data, ECT, RTT, and Down link headers being examined in our back-end code. When this is done, we generate a network quality score we can use to determine if we should intervene for a better user experience.

If it is, the score is set to 0, as we’re assuming the user wants us to do whatever is necessary to make the experience lighter and faster. If Save-Data is absent, however, we move on and weigh the values of the ECT, RTT, and Down link hints to calculate a score that describes network connection quality.

The baseline experience includes web fonts, JavaScript to drive a carousel and accordion behavior, as well as content images. When sites adapt to the information client hints provide, we don’t have to adopt an “all or nothing” approach.

Below is a Webpages waterfall of a site on a slow network that doesn’t adapt to client hints : A resource heavy site loading images, scripts, and fonts on a slow connection.

The same site on the same connection, only the “nice to have” resources are excluded in favor of faster loading. The benefits of client hints in this scenario can’t be emphasized enough and can be a serious boon to users seeking critical information over slow networks.

(Source: web.dev)

Here, “4g” represents the highest quality network connection the ECT header describes. If we initialize sect to “4g”, browsers that don’t support client hints won’t be affected.

Whenever you change a response based on an HTTP header, you need to be aware of how caches will handle future fetches for that resource. There’s a big caveat to this, though: You never want to Vary cacheable responses on a frequently changing header (like Cookie) because those resources become effectively unreachable.

Knowing this, you might want to avoid Varying on client hint headers such as RTT or Down link, because those are connection factors that could change quite often. For example, you won’t cache HTML assets if their content is dynamic, because that can break the user experience on repeat visits.

Because service workers act as proxies between clients and the servers, you have control over how resources are delivered via JavaScript. Warning: Because not all browsers support client hints, you’ll need to check what values event.request.headers.get returns.

Client hints equivalent Navigator.connection.effectiveTypeRTTnavigator.connection.rttSave-Datanavigator.connection.saveDataDownlinknavigator.connection.downlinkDevice-Memorynavigator.deviceMemory Because these APIs aren’t available everywhere you need feature check with the in operator : Service workers alone have the power to make experiences faster and more resilient due to the added ability they have to serve content when the user is offline.

(Source: www.bleepingcomputer.com)

Combined with service workers, we can create incredibly fast sites that are available offline. Consider using client hints to create truly inclusive and adaptable experiences that are aware of every user ’s device capabilities, and the networks they connect to.

Other Articles You Might Be Interested In

01: Wget Specify User Agent
02: Wget User Agent Example
03: Datapower User Agent
04: Screen Reader User Agent
05: Search Party User Agent Mac Os
06: See My User Agent
07: See My User Agent String
08: Servicenow User Agent Ews Allow
09: Set Vivaldi User Agent To Chrome
Sources
1 www.zdnet.com - https://www.zdnet.com/article/vivaldi-to-change-user-agent-string-to-chrome-due-to-unfair-blocking/
2 vivaldi.com - https://vivaldi.com/blog/user-agent-changes/
3 rootdaemon.com - https://rootdaemon.com/2019/12/19/vivaldi-to-change-user-agent-string-to-chrome-due-to-unfair-blocking/
4 www.howtogeek.com - https://www.howtogeek.com/113439/how-to-change-your-browsers-user-agent-without-installing-any-extensions/
5 helpdeskgeek.com - https://helpdeskgeek.com/how-to/how-to-change-your-browsers-user-agent-without-installing-an-extension/
6 9to5google.com - https://9to5google.com/2020/01/14/google-deprecate-chrome-user-agent-string-privacy/
7 www.guidingtech.com - https://www.guidingtech.com/use-google-docs-offline-mac/
8 sumtips.com - https://sumtips.com/software/browsers/set-user-agent-on-a-per-site-basis/