Analysis of an algorithm

January 6th, 2010 by Kevin | 4 Comments | Filed in programming

Algorithms are analyzed mainly on two categories. One is the speed of execution of the algorithm and the second is the memory requirement during the execution of the algorithm. The analysis of algorithms is important because it helps us make a decision as to which algorithm to use when we have a choice of algorithms. This is especially true in case of sorting algorithms which have a wide choice such as insertion sort, quick sort, merge sort, heap sort, counting sort, radix sort, etc.

Worst-Case Analysis
One method to analyze the performance of an algorithm is by the worst case performance of an algorithm. We can have a best case, average case and worst case analysis of any algorithm. However, the worst case analysis is the most convenient method to determine the performance of an algorithm. This is because the best case of an algorithm will not differ much from that of other algorithms, for example, in searching algorithms, the best case is that the element is found at the first search. It is quite often difficult to find the average case performance of an algorithm. The worst case of an algorithm places an upper bound on the performance of the algorithm which means the algorithm is guaranteed to perform as worse as it’s worst case and not more.

O-Notation
O-Notation is the most common notation used to express the performance of an algorithm. It expresses the worst case performance of an algorithm. An algorithms performance is measured in terms of the data that it processes.

O(1)
An algorithm is said to have a complexity of O(1) if it continues to execute in the same amount of time even with an increase in the data.

bool IsFirstElementEmpty(String[] strings)
{
   if(strcmp(strings[0], "") == 0)
   {
      return true;
   }
   return false;
}

O(n)
An algorithm is said to have a complexity of O(n) if it’s performance decreases linearly with the amount of data that it handles.

bool ContainsValue(String[] strings, String value)
{
   for(int i = 0; i < strings.Length; i++)
   {
      if(strcmp(strings[i] , value) == 0)
      {
         return true;
      }
   }
   return false;
}

O(n²)
An algorithm is said to have a complexity of O(n2) if it's execution time increases by a factor of the square of it's input data.

bool ContainsDuplicates(String[] strings)
{
   for(int i = 0; i < strings.Length; i++)
   {
      for(int j = 0; j < strings.Length; j++)
      {
         if(i == j)
         {
            continue;
         }

         if(strcmp(strings[i], strings[j]) == 0)
         {
            return true;
         }
      }
   }
   return false;
}

O(log n)
An algorithm which generally is based on a divide and conquer methodology with the division occuring at the middle of the input data set and continuing recursively is said to have a complexity of O(log n). Binary search which works by dividing the sorted data into half and working on each half recursively is said to have a complexityf of O(log n).

The following table shows how the performance degrades with the increase in the processing data of the algorithm.

constant logarithmic linear quadratic cubic
n O(1) O(log N) O(N) O(N log N) O(N2) O(N3)
1 1 1 1 1 1 1
2 1 1 2 2 4 8
4 1 2 4 8 16 64
8 1 3 8 24 64 512
16 1 4 16 64 256 4,096
1,024 1 10 1,024 10,240 1,048,576 1,073,741,824
1,048,576 1 20 1,048,576 20,971,520 1012

A table showing the performance analysis of some sorting algorithms

Type of Sort Best Worst Average Comments
BubbleSort O(N) O(N2) O(N2) Not a good sort, except with ideal data.
Selection sort O(N2) O(N2) O(N2) Perhaps best of O(N2) sorts
QuickSort O(N log N) O(N2) O(N log N) Good, but it worst case is O(N2)
HeapSort O(N log N) O(N log N) O(N log N) Typically slower than QuickSort, but worst case is much better.

*n or N is the number of elements of data that is being processed by the algorithm.

You can find more information at:

 

Related Posts:

Tags: ,

Data structures and algorithms

December 25th, 2009 by Kevin | No Comments | Filed in programming

A data structure is a method of organizing data so that it can be used in a more efficient manner. While we can build custom data structures as per our needs, some of the commonly used data structures are lists, stacks, heaps, trees, queues, hash tables, graphs and priority queues. Data structures compliment algorithms used in computer solutions. An algorithm is a sequence of procedures to be used for the solution of any problem (including computer problems). Data structures and algorithms are used together to create simple, elegant and efficient solutions to many problems which would have otherwise proven complex. The advantages of using data structures and algorithms are three fold as mentioned below:

1) Efficiency
Suppose we want to search a particular book in a list of several books in a library. We could use the normal method of placing the books in an array and checking each book in the array to see if it matches the one that we are searching. But this is a higly inefficient method of a search as the time for the search will be too high in most cases. But if we use a data structure like a hash table or a binary tree or the quick sort or heap sort algorithm, we can drastically reduce the time for the search making our solution highly efficient.

2) Abstraction
A programming language consists of several data types which can be used to solve computer problems. However, to solve complex problems, we can use data structures to enable us to think in the problem space rather than the solution space (think in terms of what to do rather than how to do). Data structures generally work with operations on them such as link lists with insert, remove, procedures creating what we know as abstract data types. An algorithm works on similar lines in the effect that we can use them with an abstract view. For example when we need to sort the employees table, we can do so with a good sorting algorithm(thinking in the problem space) and thus improving our view of the entire solution(reducing the complexity as an effect).

3) Reusability
Great emphasis is laid on reusable components in software because of the reduction in the development and testing time by using already well tested reusable modules. Algorithms and data structures being abstract components can be reused to solve several commonly occuring problems such as sorting and searching. Moreover, these have been tried, tested and immensely improved upon by developers and scientists who have worked towards creating highly efficient, reusable solutions. Further more, several complex problems after considerable thought can be broken down into smaller ones which can be solved using tried and tested algorithms which in effect makes the solution simple.

 

Related Posts:

Tags: , ,