Sunday, April 17, 2016

Install Cyanogenmod 13 on Samsung Galaxy S4


These are my installation notes from installing Cyanogenmod 13 on Samsung Galaxy S4. This S4 was originally brought from T-Mobile and its model number is SGH-M919. The Cyanogenmod codename for this device is jfltetmo. Prior to following my notes, read through the CM wiki at https://wiki.cyanogenmod.org/w/Jfltetmo_Info

1. Install JDK and Android SDK for adb

2. Install heimdall. However the classic one doesn't install. Had to use Heimdall-1.4.1-Unofficial-Signed.dmg

3. Go into Download Mode (Vol Down + Power + Home Key) with no wires connected. After you are in the loading
screen, connect USB wire to computer and install recovery image (from TWRP).

4. Install TWRP 3.0.2-0 using heimdall as per command:
heimdall flash --RECOVERY ~/Downloads/Cyanogenmod/Samsung\ S4/twrp-3.0.2-0-jfltetmo.img --no-reboot

5. In case that doesn't work, try OUDHS-Recovery-jfltetmo-1.0.3.3-recovery.img. However, this is
sticky. When using the OUDHS-Recovery-jfltetmo-1.0.3.3-recovery.img, it screws up the booting in a way that no other
recovery image (TWRP, CWM, etc.) can fix it. The best way to fix it is to use OUDHS-Recovery-Installer-Premia-4G-1.0.3.4-2.img
and then use twrp-3.0.2-0-jfltetmo.img in sequence. This allows TWRP to run without issues

6. On your SD card, copy "cm-13.0-20160417-NIGHTLY-jfltetmo.zip" and "open_gapps-arm-6.0-pico-20160417.zip".

7. Load the phone into Recovery Mode (Vol Up + Power + Home Key). TWRP allows you to install
multiple zips (up to 10). Install both the zips from #6 by selecting them in order (CM first).

8. Reboot your phone and welcome to Cyanogenmod 13 (Android 6.0.1) on your Samsung Galaxy S4 (SGH-M919)

Saturday, September 22, 2012

Booting Fedora 17 from an SD Card

After booting from the external drive (Fedora 17 Live), I mounted the ISO file in loopback mode
[root@localhost home]# mkdir /tmp/livecd
[root@localhost home]# mount -o loop /run/media/liveuser/WINDOWS/Fedora-17-i686-Live-Desktop.iso /tmp/livecd
[root@localhost home]# cd /tmp/livecd/LiveOS/
Next make sure that you have an empty SD Card at '/dev/mmcblk0'. Erase the head if need be and recreate the partition table.
## Destroys partition table and MBR
[root@localhost LiveOS]# dd if=/dev/zero of=/dev/mmcblk0 bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 21.0922 s, 5.0 MB/s

[root@localhost LiveOS]# fdisk /dev/mmcblk0
Welcome to fdisk (util-linux 2.21.1).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


WARNING: GPT (GUID Partition Table) detected on '/dev/mmcblk0'! The util fdisk doesn't support GPT. Use GNU Parted.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x4bc55751.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 
Using default value 1
First sector (2048-3994623, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-3994623, default 3994623): 
Using default value 3994623
Partition 1 of type Linux and of size 1.9 GiB is set

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): b
Changed system type of partition 1 to b (W95 FAT32)

Command (m for help): a
Partition number (1-4): 1

Command (m for help): p
Disk /dev/mmcblk0: 2045 MB, 2045247488 bytes
4 heads, 16 sectors/track, 62416 cylinders, total 3994624 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x59ff053c

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1   *        2048     3994623     1996288    b  W95 FAT32

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.
[root@localhost ~]# 
Next, create a file system on the SD Card partition as per:
[root@localhost ~]# mkdosfs -n SDCARD -F 32 /dev/mmcblk0p1 
mkdosfs 3.0.12 (29 Oct 2011)
[root@localhost ~]# 
Once the SD Card has been prepared, use the 'livecd-iso-to-disk' utility to make our live Fedora card.
[root@localhost LiveOS]# ./livecd-iso-to-disk --reset-mbr /run/media/liveuser/WINDOWS/Fedora-17-i686-Live-Desktop.iso /dev/mmcblk0p1 Verifying image...
/run/media/liveuser/WINDOWS/Fedora-17-i686-Live-Desktop.iso:   69c0ec4f4d8cd8203cb1e46a09001acc
Fragment sums: 11b5317a1a8a1cf35b9ff2184d716cae27c68d34b4c7b64f2ef714cfb399
Fragment count: 20
Press [Esc] to abort check.
Checking: 100.0%

The media check is complete, the result is: PASS.

It is OK to use this media.
WARNING: THIS WILL DESTROY ANY DATA ON /dev/mmcblk0!!!
Press Enter to continue or ctrl-c to abort

2 bytes were erased at offset 0x000001fe (dos): 55 aa
Waiting for devices to settle...
mkdosfs 3.0.12 (29 Oct 2011)
/dev/mmcblk01: No such file or directory
Cleaning up to exit...
[root@localhost LiveOS]# 
At this point, the card is bootable and ready to test.
Unfortunately, this didn't work well for me. So I followed an alternate route. I now tried to use a Mac to make the SD Card bootable.
Convert the ISO file into an IMG using the 'hdiutil' as per:
ksutaria@Downloads$ hdiutil convert -format UDRW -o ./Fedora-17-i686-Live-Desktop.img ./Fedora-17-i686-Live-Desktop.iso
Reading Master Boot Record (MBR : 0)…
Reading Fedora-17-i686-Live-Desktop.iso  (Apple_ISO : 1)…
Reading  (Type EF : 2)…
Reading Fedora-17-i686-Live-Desktop.iso  (Apple_ISO : 3)…
...............................................................................
Elapsed Time: 32.124s
Speed: 20.1Mbytes/sec
Savings: 0.0%
created: /Users/ksutaria/Downloads/Fedora-17-i686-Live-Desktop.img.dmg
ksutaria@Downloads$ mv Fedora-17-i686-Live-Desktop.img.dmg Fedora-17-i686-Live-Desktop.i
Fedora-17-i686-Live-Desktop.img.dmg  Fedora-17-i686-Live-Desktop.iso
Note that Macs tend to add ".dmg" to the file name. Fix this:
ksutaria@Downloads$ mv Fedora-17-i686-Live-Desktop.img.dmg Fedora-17-i686-Live-Desktop.img
Next, identify the right disk by using "diskutil list". Once identified use dd to write the image to the card. Note that the use of "rdisk" instead of "disk" may result in a quicker flush to disk. YMMV.
ksutaria@Downloads$ sudo dd if=/Users/ksutaria/Downloads/Fedora-17-i686-Live-Desktop.img of=/dev/rdisk1 bs=1m
645+1 records in
645+1 records out
677355520 bytes transferred in 177.251607 secs (3821435 bytes/sec)
ksutaria@Downloads$ 

Monday, April 20, 2009

Load Balancing and Scaling with Squid-HTTPD-Tomcat

One of our clients host a content management system built on top of technology I use at work. The technology is a custom server that stacks on top of the J2EE application servers like JBoss or a servlet container like Apache Tomcat. Our standard stack that we prescribe to clients is the MySQL-Apache Tomcat stack.

Over a period of few weeks, the needs of this client grew well beyond the standard stack. Considering the amount of static content being served by the Content Management System, a caching server such as Squid was a logical choice. Soon it was apparent, that this wouldn't scale either. Hence as the next iteration, we prescribed a solution with three Apache Tomcat instances running on 32-bit JVM instances of 1.5GB memory each, being load balanced by Apache HTTPD using mod-jk connnector, and front-ended with a Squid caching server. Since the public side had only read requests, a single MySQL server as the back-end was sufficient.

Being a unique stack in itself, not to mention the abundant complexities involved in between versions and dependencies, I thought it would be great to document the setup for future reference. As of now the stack is working well for the client, with the added ability to bring down one or even two Apache Tomcat instances for maintenance and leaving the site unaffected. The architecture is depicted in Figure 1 below:


The versions for softwares used are:

[root@1233797-app45 ~]# /usr/share/tomcat/bin/version.sh
Using CATALINA_BASE:   /usr/share/tomcat
Using CATALINA_HOME:   /usr/share/tomcat
Using CATALINA_TMPDIR: /usr/share/tomcat/temp
Using JRE_HOME:       /usr/java/jdk1.5.0_17
Server version: Apache Tomcat/5.5.17
Server built:   Apr 14 2006 02:08:29
Server number:  5.5.17.0
OS Name:        Linux
OS Version:     2.6.9-67.0.22.ELsmp
Architecture:   i386
JVM Version:    1.5.0_17-b04
JVM Vendor:     Sun Microsystems Inc.
[root@183787-app9 ~]# rpm -q httpd squid mysql-server
httpd-2.0.52-41.ent.4
squid-2.5.STABLE14-4.el4
mysql-server-5.0.77-1.rs.el4

Following are the steps to setup the servers with reference to Figure 1. Customize it for your local setup.

Edit the /etc/httpd/conf/httpd.conf. Change the IP and port that HTTPD listens on:
Listen 192.168.1.139:8084

Download the mod_jk connector from Apache Tomcat Connectors Download. Since I ran into issues finding pre-packaged RPM for RHEL, I used one of the binaries available from their binaries link on that page. Save the "mod_jk-*.so" file along-with other modules for HTTPD. Append the following lines to the /etc/httpd/conf/httpd.conf


LoadModule jk_module modules/mod_jk.so

JkMount jkstatus
order deny,allow
allow from 127.0.0.1
deny from all



  JkWorkersFile conf/workers.properties
  JkShmFile /var/cache/httpd/mod_jk.shm
  JkLogFile /var/log/httpd/mod_jk.log
  JkLogLevel info
  JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
  JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
  JkRequestLogFormat "%w %V %T"
  SetEnvIf Request_URI "/webmail/*" no-jk
  SetEnvIf Request_URi "/mailman/*" no-jk
  SetEnvIf Request_URI "/awstatsclasses/*" no-jk
  SetEnvIf Request_URI "/awstatscss/*" no-jk
  SetEnvIf Request_URI "/awstatsicons/*" no-jk
  SetEnvIf Request_URI "/awstats/*" no-jk
  JkMount  /* router
JkMount /jkmanager/* jkstatus

Create a /etc/httpd/conf/workers.properties file as shown under:


# Define some properties
workers.apache_log=/var/apache/logs
workers.tomcat_home=/usr/share/tomcat
workers.java_home=/usr/java/jrockit-R27.4.0-jdk1.5.0_12
ps=/

# The advanced router LB worker
worker.list=router,tomcat1,tomcat2,tomcat3,jkstatus

# Set properties for tomcat1 (ajp13)
worker.tomcat1.type=ajp13
worker.tomcat1.host=192.168.1.140
worker.tomcat1.port=8010
worker.tomcat1.lbfactor=1

# Set properties for tomcat2 (ajp13)
worker.tomcat2.type=ajp13
worker.tomcat2.host=192.168.1.140
worker.tomcat2.port=8011
worker.tomcat2.lbfactor=1

# Set properties for tomcat3 (ajp13)
worker.tomcat3.type=ajp13
worker.tomcat3.host=192.168.1.140
worker.tomcat3.port=8012
worker.tomcat3.lbfactor=1

# Define the LB worker
worker.router.type=lb
worker.router.balance_workers=tomcat1,tomcat2,tomcat3

worker.jkstatus.type=status


On each of the workers (tomcat1,tomcat2,tomcat3) make the following changes to the /usr/share/tomcatXX/conf/server.xml file where XX indicates the number of tomcat:


  • Change the connector configuration so that the port numbers are 8081, 8082, 8083 for tomcat1, tomcat2 and tomcat3 respectively. This helps validate individual tomcat instances.

      
    
    
  • The mod_jk connector talks to the worker instances on the AJP connector port. This port can be specified in the server.xml file as shown below. We picked ports 8010-8012 for tomcat1-tomcat3

        
    

  • Uncomment the lines that set the jvmRoute or append if necessary. Note that the value for the jvmRoute should be the same as that used in the workers.properties file for the respective tomcat instance.

    
    
    

  • To change the root context ensure that the “Context” directive is updated as under:

    
    
         
         
         
      
    
Restart the Apache HTTPD Server and the individual Apache Tomcat Servers and validate the Load-Balancing configuration works as expected. Use the following log files to isolate the problem if any:

tail -f /var/log/httpd/access.log
tail -f /var/log/httpd/mod_jk.log
tail -f /usr/share/tomcatXX/logs/catalina.out
tail -f /var/log/httpd/error_log
Assuming the load balancing setup is working as expected, lets move on to the Squid as a Reverse Proxy. There have been changes between versions 2.5 and 2.6 that affect the configuration file options for the setup. Following are the instructions for version 2.5. Set the IP-Port on which Squid listens. Next, configure the cache-peer of Squid to be Apache HTTPD at port 8084.

http_port 192.168.1.139:80
cache_peer 192.168.1.139 parent 8084 0 no-query
Since, we wanted to cache the pages with parameters as well, we commented the no_cache option. The hierarchy stoplist option was also commented out as shown.

#hierarchy_stoplist cgi-bin ?
#acl QUERY urlpath_regex cgi-bin \?
#no_cache deny QUERY
Depending on your cache size requirements, tweak the following parameters:

cache_mem 512 MB
maximum_object_size 8192 KB
maximum_object_size_in_memory 256 KB
cache_dir ufs /var/spool/squid 1024 16 256
Define access control lists for the destination and trusted networks. This is followed by http_access rules that allow or deny as per security requirements. Note how the rule 'filter_admin_portal' uses a regular expression match to filter access for the admin portal.

acl our_networks src 192.168.1.0/24 10.20.30.0/24
acl dst_mywebapp dst 192.168.1.0/255.255.255.0
acl filter_admin_portal urlpath_regex -i ^/[^/]*admin[^/]*
http_access deny filter_admin_portal
http_access allow dst_mywebapp
http_access allow our_networks
http_access deny all
Squid needs to know the 'Real' HTTP server for which it acts as an accelerator. The parameters httpd_accel_host and httpd_accel_port should point to the Apache HTTPD server setup earlier.

httpd_accel_host 192.168.1.139
httpd_accel_port 8084
httpd_accel_single_host on
httpd_accel_with_proxy on
Once Squid has been restarted, the entire setup should be ready for some testing. Use the log files described above and the following log files to iron out discrepancies, if any.

tail -f /var/log/squid/squid.out
tail -f /var/log/squid/access.log
tail -f /var/log/squid/cache.log
Along the way, we used JMeter for load/stress testing, JConsole for viewing the JVM health in real time, and WireShark/Fiddler to look at HTTP headers and packets. These tools helped find find bottlenecks, tweak parameters and re-architect the solution.

Although, there are multiple points of failure in this architecture, the primary aim was load balancing. Fault tolerance and reliability weren't the primary concerns at this point of time. In a future iteration, the plan is to address these concerns as well.

Tuesday, April 7, 2009

Corrupt RPM Database

The other day I was on a client machine which happened to give me the following exception when looking up an RPM package using yum.


[root@staging ~]# yum search vnc
Loading "rhnplugin" plugin
rpmdb: Lock table is out of available locker entries
rpmdb: Unknown locker ID: 8470
error: db4 error(22) from db->close: Invalid argument
error: cannot open Packages index using db3 - Cannot allocate memory (12)
error: cannot open Packages database in /var/lib/rpm
Traceback (most recent call last):
File "/usr/bin/yum", line 29, in ?
yummain.main(sys.argv[1:])
File "/usr/share/yum-cli/yummain.py", line 85, in main
base.getOptionsConfig(args)
File "/usr/share/yum-cli/cli.py", line 163, in getOptionsConfig
disabled_plugins=self.optparser._splitArg(opts.disableplugins))
File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 164, in _getConfig
self._conf = config.readMainConfig(startupconf)
File "/usr/lib/python2.4/site-packages/yum/config.py", line 685, in readMainConfig
yumvars['releasever'] = _getsysver(startupconf.installroot, startupconf.distroverpkg)
File "/usr/lib/python2.4/site-packages/yum/config.py", line 755, in _getsysver
idx = ts.dbMatch('provides', distroverpkg)
TypeError: rpmdb open failed


Apparently, this is the case of a corrupt RPM database and can be fixed by rebuilding the database as follows:


[root@staging ~]# rm /var/lib/rpm/__db.*
rm: remove regular empty file `/var/lib/rpm/__db.000'? y
rm: remove regular file `/var/lib/rpm/__db.001'? y
rm: remove regular file `/var/lib/rpm/__db.002'? y
rm: remove regular file `/var/lib/rpm/__db.003'? y
[root@staging ~]# rpm --rebuilddb

Monday, February 2, 2009

Check Available Disk Space and Notify

Here's some handy bash script to notfiy support when we are running low on disk space.


#!/bin/bash
#
# Description:Checks the available space for the disk mounted as / (root)
# If disk space is less than 10GB it will send out a notification
#

# Send emails to these people
EMAILID=support@mydomain.com

# Min Free Space
MINSPACE=10485760

# Get the space available
FREESPACE=`df / | awk '{ if( NR == 2) print $4 }'`

# Send email if required
if [ $FREESPACE -le $MINSPACE ];
then
logger "Warning: Root Drive Running Out Of Disk Space [${FREESPACE}KB]"
mail -s 'My Favourite Production Server Disk Space' ${EMAILID} << EOF
My Favourite Production server root drive is running out of disk space. Current free space is ${FREESPACE} KB @ $(date)
EOF
fi

Friday, May 23, 2008

Mantis + LDAP Authentication

Mantis is one of the many bug tracking softwares available out there (Refer: Top Configuration Management). For an internal project at work, I needed a bug tracker and Mantis was the logical choice due to prior experience within the development team. The machine at hand was running on CentOS Release 4.4 and required the following rpm packages:



[root@eclipse html]# rpm -q httpd mysql-server php php-mysql php-ldap

httpd-2.0.52-28.ent.centos4

mysql-server-4.1.20-1.RHEL4.1

php-4.3.9-3.22.9

php-mysql-4.3.9-3.22.9

php-ldap-4.3.9-3.22.9


There were issues with using the community-server rpms for MySQL 5.0.51 as the php-mysql and php-ldap packages weren't compatible. Since, this was going to be an internal bug tracker, and we weren't expecting to use any of the 5.0 features, it was a fair compromise for the power of Apache-PHP-LDAP-Mantis.



We have an existing LDAP server which stores the authentication credentials companywide. Out of the box, Mantis allows for new user sign-up but this would mean that every user in the organization would have to keep track of another set of credentials. Hence, LDAP integration was a must. I expected LDAP authentication to work out of the box but it turned out to be a little more involved. I am a novice when it comes to PHP and it took a little bit of searching and reading manuals to wade my way through the Mantis-PHP maze of code.



In order to help people who want to walk down the same path, I aim to simplify their life by providing steps to achieve this feat:



1. Download Mantis and unzip at /var/www/html (or whatever your DocumentRoot is at in /etc/httpd/conf/httpd.conf) and rename the folder to mantis. As of writing this post, the stable version was mantis-1.1.1



2. A bunch of global Mantis settings are listed in the mantis/config_defaults_inc.php. I tried to override the ones that were relevant to this integration. Here's the file:


# --- database variables ---------

# set these values to match your setup
$g_hostname = "10.0.0.4";
$g_db_username = "mantisuser";
$g_db_password = "password";
$g_database_name = "mantis";
$g_db_type = "mysql";

# --- email variables -------------

$g_administrator_email = 'admin@mycompany.com';
$g_webmaster_email = 'webmaster@mycompany.com';

# the "From: " field in emails
$g_from_email = 'noreply@mycompany.com';

# the return address for bounced mail
$g_return_path_email = 'admin@mycompany.com';

# --- file upload settings --------
# This is the master setting to disable *all* file uploading functionality
#
# The default value is ON but you must make sure file uploading is enabled
# in PHP as well. You may need to add "file_uploads = TRUE" to your php.ini.
$g_allow_file_upload = ON;


# LDAP
$g_login_method = LDAP;
$g_ldap_server = 'ldap://10.0.0.4/';
$g_ldap_port = '389';
$g_ldap_root_dn = 'dc=mycompany,dc=com';
$g_ldap_organization = 'objectClass=*'; # e.g. '(organizationname=*Traffic)'
$g_ldap_uid_field = 'uid'; # Use 'sAMAccountName' for Active Directory
$g_ldap_bind_dn = 'uid=root,ou=Users,dc=mycompany,dc=com';
$g_ldap_bind_passwd = 'cleartextsecret';
# Should we send to the LDAP email address or what MySql tells us
$g_use_ldap_email = OFF;

# The LDAP Protocol Version, if 0, then the protocol version is not set.
$g_ldap_protocol_version = 3;

# --- signup ----------------------

$g_allow_signup = OFF;
$g_lost_password_feature = OFF;


3. In the file 'mantis/core/authentication.php', change the lines in the function 'auth_attempt_login' begining "if ( false === $t_user_id ) {" as follows

if ( false === $t_user_id ) {
  if ( BASIC_AUTH == $t_login_method \\ LDAP == $t_login_method ) {
    # attempt to create the user if using BASIC_AUTH
    if ( BASIC_AUTH == $t_login_method) {
      $t_cookie_string = user_create( $p_username, $p_password );
    } elseif ( LDAP == $t_login_method ) {
      $t_cookie_string = user_create( $p_username, '', 
        ldap_email_from_username( $p_username ) );
    }
    if ( false === $t_cookie_string ) {
      # it didn't work
      return false;
    }


4. Restart your Apache HTTPD server and validate.

Wednesday, April 9, 2008

Truncate a file

While setting up Apache HTTP Server to rotate logs I came across an interesting problem. How would you truncate a file? The trivial solution that I came up with was


[root@lambda ~]# rm -f /var/log/httpd/foo.log
[root@lambda ~]# touch /var/log/httpd/foo.log


Unfortunately this doesn't work. The HTTP Server ceases logging to the file unless there is a server restart. After prying around the web I came across some interesting tricks but the one that stood out was:


[root@lambda ~]# :> /var/log/httpd/foo.log