Making operating system more secure

Mohammad M Rahman
6 min readAug 7, 2022

Operating system (OS) security refers to procedures and controls that can guarantee the confidentiality, availability, and availability (CIA) of operating systems. The purpose of OS security is to defend the OS against a variety of dangers, such as misconfigurations, remote intrusions, and malicious software like worms, trojan horses, and other viruses. The adoption of control strategies that can shield your assets against unwanted addition, deletion, and theft is often part of OS security. The use of antivirus software and other endpoint protection tools, routine OS patch updates, a firewall for observing network traffic, and enforcement of secure access through least privileges and user controls are among the most often used methods for protecting operating systems. For example, Microsoft Defender Antivirus is an antivirus application that is a part of Windows Security.
Can we enhance OS security over the basic provisions that are included with the shipment? Yes, we can from various perspectives of CIA and on the nature of protection needed such as protection against malware, DDOS attack, network intrusion, and buffer overflow. We can use the following protocols to enhance OS security:

• Through the use of virtualization, you may efficiently separate hardware and software. The fundamental benefit of virtualization is that it increases security coverage while introducing a high level of efficiency and flexibility. Virtualization can take many different forms, such as desktop, application, network, server, storage, and OS virtualization. A type of sandboxing is the virtualization of an operating system. The actions that each type of VM is capable of being constrained by design. Any additional action is prohibited. This maintains environmental security. Users are effectively isolated by the hypervisor, which runs beneath the device’s OS and divides it into several virtual machines (VMs) running locally with their own operating systems. The devices stay secure since the users are segregated. This guarantees that employees and outsiders can access firm resources without putting them in danger. The fact that none of the virtualized environments may directly access the network is another significant benefit of OS virtualization. Instead, a hidden, virtualized network layer that conducts network segmentation on the endpoint device itself provides connectivity.

• Testing for vulnerabilities in an operating system involves looking for flaws that might be hiding there. To better understand the risk to your system, identify vulnerabilities so that you can also discover potential attack vectors. By locating, categorizing, and prioritizing vulnerabilities based on their severity and effect as part of a continuous process, vulnerability assessment works to keep on top of newly discovered vulnerabilities. Typically, this procedure mixes automated technologies with manual operations. The following are some standard techniques for determining OS vulnerability:
o checking for known weaknesses
o checking the operating system’s programs and software
o detecting malware
o Checking for any unapplied fixes or updates
o patch evaluation
o port checking

• In order to determine how an attacker can successfully exploit vulnerabilities in the system, penetration testing, also known as pen testing, is a security assessment approach. The penetration testing technique simulates an exploit to assess the security of the system. Penetration testing aims to find vulnerabilities that are less evident and assists in finding potential attack vectors. Insights gained from pen testing can be used by security teams to implement efficient security solutions. There are three forms of penetration testing, and each one offers various perspectives on the operating system’s security and exploitation potential:
o White Box: The penetration tester is fully knowledgeable about the technical aspects of the system under test.
o Grey Box: The pentester’s technical understanding of the system under test is limited.
o Black Box: The system being tested is unknown to the pentester on a technical level.

• Give users read-only access to the necessary directories. The user permissions are available to attackers who gain access to a program. Access is default denied. Except for users to whom access is expressly allowed, access to resources is forbidden to all other users. You can prevent any user from having read and write access to any directory structures. Access to the directories and files is only available to users who have been specifically granted these permissions. Additionally, any resources that a manager failed to notice are safeguarded by this policy.

• To guarantee the integrity of the system, construct production systems from a known and repeatable procedure. Periodically compare systems to snapshots of the original system. To verify the integrity of the system, use any accessible third-party auditing tools. Regularly create a system resource backup. Record all security-related activities, such as successful and unsuccessful logons, logoffs, and modifications to user permissions. keep an eye on the system logs. To correlate time for forensics, use a time server. By limiting who can access the system log files, you can secure them. Logs are crucial for routine upkeep and as a tool for catastrophe recovery. They must therefore be shielded from user interference and system failures. Protect the configuration file for logging. The configuration file contains options that, if modified, could jeopardize the log system’s dependability. For instance, wrongly setting the log level may prevent some failures from being recorded. Turn on Web server logging for access requests. This may be helpful in spotting malevolent behavior.

• On the server machine, run the bare minimum of necessary services. Just the services you require to run the application should be used. Every service could serve as the starting point of a malicious assault. Your system becomes easier to administer when fewer services are active. Lower the level of network service users’ access permissions. Make sure that none of the user accounts with access to the Web server may access the shell functions. On Microsoft Windows operating systems, make sure that no unnecessary services are running and that they are not set to launch automatically. Verify that the necessary UNIX services are up and working. Make use of wrapper services like iptables. By often checking for security updates, make sure the services are up to date. If at all possible, stay away from using services using graphical user interfaces (GUI). These services expose numerous well-known security flaws. The number of trustworthy ports listed in the /etc/services file should be decreased. To remove potential access points to the system, delete or comment out the ports that you do not intend to utilize. Through a system of secrets and keys, cryptography can assist with both the challenges of trust and confidentiality. The recipient in the latter instance holds the key, ensuring that only the intended recipient can accurately receive the message. In the former case, the key is retained by the sender, ensuring that the recipient knows that only the genuine author could have sent the message. Keys must be carefully secured because they are made to be impossible to deduce from any publicly available information. Encoding a message so that only the intended receiver can decode and read it is the fundamental concept of encryption.

The answer to the question of “are there any aspects that you think will make your OS design more secure than today’s OSes” is difficult because one should know the architecture and source code of the major OS to really propose effective new cutting-edge security solutions which may also deal with zero-day attacks. So even though OS can be made more secure than today’s OS it needs detailed study first. One possible proposal could be an artificial intelligence-based security solution where AI detects potential attacks before actually happening. This can be termed a preemptive strike.

(Hysolate, n.d.; Microsoft, n.d.; Operating Systems: Security, n.d.; Techniques for Securing the Operating System, 2014)


Hysolate. (n.d.). Understanding OS Security: Threats and Security Controls. Hysolate.

Microsoft. (n.d.). Stay protected with Windows Security.

Operating Systems: Security. (n.d.).

Techniques for Securing the Operating System. (2014, August 26).



Mohammad M Rahman

Research interest: Islam, Computer science, Psychology/Sociology. Please see my profile links for further info.